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FOREWORD 


E valuation of educational achievement is so complex a field 
that even test experts are humble in the presence of the many 
complications and constantly seek to improve their under- 
standing as well as their techniques. The need to measure as 
surely as we can the effects of our educational programs is a com- 
pulsion felt by all educators and administrators. Any new light 
on this problem is therefore a stimulus and a challenge. The 
Cooperative Study of Evaluation in General Education was spon- 
sored by the American Council on Education with the objective 
of discovery, and with the hope that the results would be useful 
to teachers, especially those in general education. 

This volume is a report of the Director of the Cooperative 
Study to the Committee on Measurement and Evaluation, the 
committee appointed by the Council to give continuing study to 
evaluation and testing needs and problems in higher education. 
The Cooperative Study of Evaluation in General Education has 
been the major work of this committee up to the present. The very 
title of the report, with its emphasis upon the exploratory nature 
of the 'ivork ivhich was done, suggests that we have far to go in 
finding ways of appraising more surely what needs to be done in the 
classroom. One of the appealing aspects of this study is that a large 
number of classroom teachers from nineteen colleges and univer- 
sities of many kinds provided the working forces which carried 
out the major planning and research. 

The Cooperative Study ivas made possible by the generous 
assistance of the Carnegie Corporation of New York and by annual 
contributions from the participating institutions. The Council 
has also participated in the financing. All of the participating 
institutions continued without interruption their support of tlie ' 
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Study This is testimony to the significant objectives of the Study, 
the leadership of the Director and his staff, and the high caliber of 
the representatives of the nineteen colleges 

Exploration is a word which suggests the future By their otvn 
testimony, those who had a part m the Cooperative Study are giving 
more attention in their teaching to objectives and to appraisal of 
progress toward objectives, and we may hope that this concern will 
continue in these many centers of inttiati\c It is the hope of the 
Director of the Cooperative Study and of the Committee on Meas 
urement and Evaluation that this report will have similar effects 
in many institutions that did not enjoy die advantages of active 
participation 

Arthur S ADA^^s, President 
American Council on Education 



PREFACE 


T his volume, the final report of the Cooperative Study of 
Evaluation in General Education, is a composite product 
undertaking several different tasks. It presents a brief de- 
scription of the way the Study has operated. It reports on some of 
the research conducted by the several committees of the Study. It 
deals with some of the technical problems of educational testing 
and indicates some of the contributions which the project may 
have made toward their solution. Being a report witten entirely 
by the Director and Assistant Director of the Study, it also involves 
a distinguishable, although not necessarily distinctive, point of view 
regarding general education and the role of evaluation in educa- 
tion. Although we have attempted to discuss the activities and the 
research results objectively, our own philosophies are implicit in 
our approach to the problems. 

It was originally hoped that the Study, initiated in the spring of 
1950, would cover a five-year period embracing (1) two years 
to study objectives, to develop evaluation procedures, and to plan 
a research design, (2) two years to follow groups of freshmen from 
the beginning of their first year to the beginning of their third year, 
(3) a final year to make statistical analyses of results, to investigate 
implications, and to prepare the final report. Circumstances en- 
forced a modification of this whereby stages 1 and 2 were 
assigned to a three-year period of active endeavor and a final six 
months was employed for stage 3. Although this reduced length 
resulted in inability to carry out the two-year follow-through of 
freshmen, the actual nature of the project as it developed made 
this omission relatively unimportant. 

A great deal was accomplished in regard to instructional prob- 
lems, much more, in fact, than it has been possible or appropriate 
ix 
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to report in this volume The various committees found that all 
too little IS being done m most general education courses to pro 
mote the progress of students toward tlie objectives inherent m 
general education Indeed, it may be that the contributions to 
instructional problems will be more significant than the tests and 
the statistical data obtained through their use Considered from 
this point of view, the reduced length of the project was less 
significant than it would othemise appear to be After all, long 
term studies of student development cannot be of great value until 
more serious attention is given to relating classroom practice to 
the avowed educational objectives 

The participating institutions were assured that no comparisons 
among institutions would be repotted in identifiable form Hence, 
the authors have identified colleges and universities only in a few 
cases where the schools themselves have previously distributed re 
ports containing the data The anonymous quality of the data may 
detract from the interest of the report for some readers, but it also 
makes possible comparisons of attributes of courses and programs 
which could not be made othenvise 
If It IS true that general education courses and programs fre 
quently do not live up to expectations, as this report brings out in 
places, there is also much to inspire optimism The mere fact that 
so many individuals would attend so assiduously to a study of 
evaluation in general education is one proof of the strength of the 
movement Few general education teachers are satisfied with what 
they ate presently doing and they are constantly seeking for neiv 
and better ways We have hopes for significant developments in 
the future and we would like to believe that this report, by its 
realistic summary of present practice, may serve in some slight 
measure to expedite such developments 

Paul L Dressel, Director 

Lewis B Mayhew. Assistant Director 
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WHY AJV EVALUATION STUDY? 


1 


T R McConnell has observed that general education “is a 
movement which began as a re examination of the nature 
and purposes of liberal education and which is leading 
toward a revitalization of the liberal arts, and perhaps to a com 
plete reconsideration of the nature of the learning process ’ ^ The 
word “movement’ is well chosen, for the major common element 
of various approaches to general education is to be found in the 
reaction against the compartmemalization and ovetspeciahzation 
permitted and even encouraged by the free elective system and the 
equally serious narrowness of trainirlg currently required m many 
fields of technical or professional perparation This common ele- 
ment of reaction is scarcely sufficient to provide any clear or gen- 
erally acceptable indication of the direction of development Gen 
eral education is not, therefore, a single, easily recognizable 
phenomenon but, rather, it has a diversity of forms This diversi- 
fication IS demonstrated in the courses, in the combinations of 
courses, m the subject matter, in the methods of instruction, in 
the types of administrative organization, and even, though to a 
lesser extent, m the objectives accepted 

In Higher Education for American Democracy we find that 
“ ‘General education’ is the term that has come to be accepted for 
those phases of nonspecialized and nonvocational learning which 
should be the common experience of all educated men and 
women Yet on another page of the same report, italicized for 
emphasis, is found ‘ In the future as in the past, American higher 

*T R McConnell General Education An Analysis in The Fifty Ftrst yearbook 
of the National Society for the Study of Education, 1952, Part J— General Education 
(Chicago Univenity of Chicago Press 1952) p I 
*Ii\gher Education for Amertcan Democracy The Report of the Presidents 
Commission on Higher Education, Vol J-^Eslabltshing the Coals (Washington 
Govemmcnt Pnnung OIScc andNewYoik Haiper&Bros.. 19'17),p 49 

1 



2 GENERAL EDUCATION 

education wiU embody the principle o£ diversity m unity each 
institution. State or other agency, will continue to make its own 
contribution in its own way But educational leaders should try to 
agree on certain common objectives that can serve as a stimulus 
and guide to individual decision and action ' * The italicizing o£ 
the phases common experience” and ' common objectives ’ is ours, 
and serves to focus attention on these two elements o£ general 
education -which receive various interpretations and emphases 
which in turn result in quite different programs of general educa 
tion Experiences and objectives are by no means equivalent, it is 
undoubtedly possible to attain the same objectives by various 
experiences, but the expenences as the immediate and obvious 
elements of the educational program may easily supplant the 
objectives and become ends in themselves Accordingly, general 
education programs may differ by virtue of difference in objectives, 
they may also differ, despite acceptance of common objectives, in 
the range of expenences which are provided as a means for the 
facilitation of the achievement of these objectives by the students 
Appreciation of the difficulties of evaluation in general education 
requires some examination of the nature of these differences, for 
ultimately they reflect basic, though often implicit, philosophical 
differences as to the nature of education 

The Objectives of General Education 

We have previously noted that, superficially at least, there is less 
disagreement about general education objectives than about the 
means of achieving them The objectives found m A Design for 
General Educalton* General Education in Action,^ and the Report 
of the President’s Commission are illustrative of the better and 
most commonly accepted statements Most of the eleven objectives 
stated in Higher Education for American Democracy^ are palat 
able in some degree to most general education faculties Neverthe 
less, a survey of some twenty colleges during the formative stages 
of the Cooperative Study of Evaluation in General Education 

* Ifcjd , p 6 

..".i ('Vitthington Aaitnan Council on Eduealion 

IMl) pp 31-50 

•n JoWu CcumI EJncnl,™ del, on (Waihmgion American Coun 

cil on Education 19:>2} pp 19-32 

*Jltgher Eduealion Jor Ammean Democracy, pp 50-58 
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showed that in only five or six o£ these were objectives receiving 
anywhere near unanimous endorsement Maintenance and im- 
provement o£ health, acquiring the knowledge and attitudes basic 
to a satisfying family life, and the choice of a vocation, were three of 
the objectives that were frequently rejected as of major concern. 
While some general education programs professed no concern at 
all with these objectives, others accepted them as secondary or pos- 
sibly ultimate goals which might be possible and indirect results of 
the attainment of other and more immediate objectives. There was 
also evidence of considerable disagreement among members of a 
given general education staff as to the range of objectives accepted 
for the program and a tendency to associate only one or two of the 
objectives with tlte particular course taught. 

Thus, a set of general eduction objectives may not describe the 
foci of attention of all of a general education staff, but rather may 
be an accumulated listing of individual course objectives with 
perhaps a few others added to make a better-rounded description 
of the generally educated person. The list may represent adminis- 
trative or faculty committee thinking rather than the consensus 
of a total staff. 

The range of objectives accepted by a program or by an in- 
dividual has some implications as to nature of the educational ex- 
periences provided. Rejection of objectives involving mental and 
physical health, vocational choice, and personal, social, and family 
adjustment, usually characterizes a program in which the courses 
are devoted to cultivation of the intellect and to the provision of 
an overview of the more significant aspects of the cultural heritage 
as embraced in science, social science, and the humanities. Accept- 
ance of objectives in the affective domain or of the life adjustment 
type does not, on the other hand, necessarily involve any direct 
counterpart in courses or in extracourse educational experience. As 
Sidney Hook has noted, there is a tendency "to rest content with 
the formulation of goals, to overlook the difficulties in implement- 
ing them in concrete curriculums, and the still greater difficulties 
of evaluating to what extent we make good what our college 
catalogs promise."^ Agreement on objectives, even when it is 

* Sidney Hook, "PeTcnDial and Temporal Goals in Education/’ Journal of Higher 
Education, January 19S2, p. 12. 
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achieved, means little, for the stating of objectives is really in- 
separable from the decision as to the means by which these are to 
be achieved. 

Variations in Eddcationai. 

College faculty members have a high degree of autonomy in the 
handling of their classes. Thus, it becomes impossible to character- 
ize the instruction in a general education program or even in a 
single course unless only one instructor is involved. Even in this 
situation, the interaction of the instructor with individual students 
may result in differences in the educational experiences of in- 
dividuals within the same class. Therefore, any characterixation of 
types of educational experience must be an oversimplification. Yet, 
for convenience here and for later reference, we shall strive to 
describe three fairly distinctive types of educational experiences 
afforded to students. 

In the attempt to provide a common experience to students, one 
or more each from science, social science, and humanities teachers 
chosen from several related traditional departments of specializa- 
tion are frequently asked to organize a broad interdepartmental 
course. With or without formally stated objectives, the result may 
be simply a course inclusive of what is construed to he the essen- 
tial knowledge in the broad area which should be in the possession 
of every generally educated person. Such courses, both by reason 
of the importance attached to the material thus selected and by 
reason of the extensiveness of the material to be covered, may and 
often do become entirely content-oriented. The function of the 
educational program, then, is to expose the student to this ac- 
cumulated wisdom which he is expected to master. There is likely 
to be little concern and no adaptation to individual differences. 
Rele\'ancy to current and future individual and societal needs is 
assumed, but the detection of the relevancy and the consequent 
application of the knowledge may be left to the student. The 
materials of the course become the ends, and the achievement of 
studenu is judged solely by their mastery of these materials. Con- 
tent-oriented instruction frequently recognizes that other objec- 
tives and needs exist, but is inclined to assign responsibilities for 
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them to otlier social agencies or to the personnel staff and the 
extracurricular aspects of higher education. Another assumption 
is that knowledge in itself is both the prerequisite to and the 
promoter of broader personal development. The phrase “content- 
oriented” is a reasonably adequate description of such an ap- 
proach. 

A somewhat different approach to course organization accepts 
more direct responsibility for the development of a number of in- 
tellectual skills which involve application of what is being learned 
to current and probable future problems. Ability to think critically 
about current social problems and ability to read current popular 
or semipopular scientific articles are typical of objectives which 
may be regarded as sufficiently important to provide criteria for 
the selection of materials to be included in the course. Relevancy 
of the material to the interests of students and to issues of the day 
may become a consideration in deciding what to teach just as much 
as the intrinsic worth of a particular segment of knowledge. Cover- 
age of particular topics may even be sacrificed to provide for a 
number of exploratory experiences for the students that are de- 
signed to provide each individual with actual practice in attacking 
and thinking through problems for himself. Teachers accepting 
this point of view argue that retention, integration, and utilization 
of knowledge will result only as the student is given opportunity 
to discuss, to analyze, and to reach his oivn conclusions in regard 
to ideas or problems not previously digested by the instructor or 
the textbook. There is more concern about the exposure of the 
student to a number of points of view rather than to one accepted 
by the instructor or textbook author. Hence, field trips, discus- 
sions, and other group or individual projects become equal in im- 
portance to lectures and textbook assignments. Attitudes, values, 
personal adjustment, and similar objectives implied by these may, 
however, be ignored or become matters of concern only ■when they 
interfere with the intellectual development of the individual. 
Responsibility for such matters is delegated to extracurricular 
agencies, and the major educational experiences are still viewed 
as supplied within the framework of the formally organized 
courses. Greater emphasis on relevance and the explicit attention 
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to intellectual objectives differentiates such an approach from the 
content-oriented one and justifies its designation as “intellectually 
oriented.” 

The third approach to general education has already been im- 
plied in the foregoing remarks. It may be characterized as student- 
oriented in that cognitive objectives are given no priority over 
affective ones. The tendency is to give priority to the total develop- 
ment of each individual and to consider all aspects of experience 
which are or can be provided by a college as of equal value insofar 
as they have significance in promoting the development of in- 
dividuals in reference to a wide range of objectives. Ideally, the 
needs, interests, and desires of each person are taken into considera- 
tion in planning the particular set of experiences of most value to 
him. Relevancy of materials and experiences to the needs of the 
individual and to the total development of that individual be- 
comes the prime consideration. Knowledge of content and intel- 
lectual development are not ignored, but the former is seen as a 
means to an end and the latter is no more important than other 
aspects of individual devcloptnctit. Whereas both of the previous 
two orientations may place great emphasis on the provision of 
common classroom experiences, the tendency in the student- 
centered orientation is to feel that no experience can mean quite 
the same thing to two different individuals Emphasis is accord- 
ingly placed on the availability of a number of different courses or 
at least on the inclusion in each course of a number of means for 
individual adaptation such as acceleration, individual projects, 
special sections, and the like. The contrast between this viewpoint 
and the content-oriented one entitles the latter to the description 
“student-oriented.” 

No general education staff is in complete unanimity on any one 
of these three approaches nor, for that matter, are individuals con- 
sistent in their adherence to one of them. The consequence is that 
almost every general education program exhibits in microcosm 
the same inconsistencies, discrepancies, or disagreements that are 
found among general education programs in different schools. The 
differences are less profound than might be expected, however, 
because the instructional staff, faced with the necessity of offering 
a definite program to students and of exhibiting at least superficial 
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harmony to the public, tends to regress to the grounds provided 
by its common concerns and by the similarity in training of its 
members. Thus, the selection and organization of content ma- 
terials based presumably on the intrinsic worth of these as judged 
by staff consensus is the typical basis for most general education 
programs. “Content-oriented” is probably not unfair as a charac- 
terization of most general education instruction, although the 
actual intent of an increasing number of teachers and courses is an 
intellectual orientation. Relatively few programs, courses, or 
teachers are to be classified as student-oriented if they are to be 
judged by actual practice, although a significant proportion of 
general education instructors are strongly sympathetic to this 
point of view. 

Need for Experimentation and Evaluation 
The varying interpretations of a given objective, the varying 
roles that objectives play in a program, and the very real difficulty 
faced in the implementation of certain objectives, all conspire to 
make general education programs and the statements of general 
education objectives less interrelated than might be expected. 
Yet similarity in objectives does exist and constitutes, on the one 
hand, one of the strongest bonds relating othenvise virtually dis- 
similar programs and, on the other hand, perhaps the clearest 
basis for distinction between a general education course or pro- 
gram and the specialized departmental courses for which the 
major aim is preparation for further work in the same field. In- 
deed, the existence of, and the challenge presented by, this com- 
monality of objectives was the major reason for the development of 
the Cooperative Study of Evaluation in General Education. 

The nature of the challenge is twofold. First of all there is the 
question as to whether general education ventures achieve their 
accepted objectives to any greater extent than do the courses which 
they seek to replace. Even the most specialized course may and 
probably does make some contribution to general education, and 
the instructors of such couises frequently believe that they make 
as much contribution to the general education of the student as 
does the course developed for this specific purpose. There is not 
and there cannot be any single answer as to w? cduca- 
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tion courses are superior to departmental courses The vanations 
in programs, courses, and instruction suggest that an answer can 
be approximated only for a particular program or type of program 
and that, e\en then, the nature of the answer may vary con 
siderably for individual instructors in the program The difference 
betiveen a content oriented general education course and a tradi 
tional departmental course must be primarily in terms of the dif 
ference in the content or matenals studied A judgment as to the 
relative efficacy of tivo such approaches must be based primarily on 
a judgment of the intrinsic worth and relevancy of the content 
Such judgments are certain to differ widely 
Comparison of educational programs which differ in objectives 
requires that any consideration of their respective merits include 
not only evidence on the achievement of objectives but a judgment 
as to the worth and the appropriateness of these objectives Such 
judgments involve philosophical differences about which no objec 
live evaluation can be made The evaluation of all of general 
education m comparison with specialized courses is, practically 
spMking an impossible task and certainly an unprofitable one 
The second aspect of the challenge presented by the com 
monahty of some objectives in general education rests m the diver 
sity of means used to accomplish the same ends The nature of this 
challenge ww clearly underscored by the remarks of Dr Earl 
^ ^ ^ degree these developments knoivn as 

wnhT .f of “ pnon reasoning 

to rteV ’ " '’‘“Pfons little attempt has been 

sTibte i m? "-hether one arrangement o£ 
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eral education, whether in content, courses orgrn.ration,”orm®od"e 
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of instruction, can be equally efficacious in producing student 
development svith regard to even those objectives commonly ac- 
cepted in the various programs. Yet differences in teachers and in 
students make it doubtful that a simple answer regarding such 
issues is possible. 

It may be unwise, and it is certainly unnecessary, to imply that 
general education should be placed on trial. If the major aims of 
general education are accepted as desirable, it is really the means 
to these ends that are to be studied and evaluated. With the term 
general education applied to a wide variety of educational ex- 
periences from rechristened but still inviolate liberal arts pro- 
grams, through vocational and homemaking courses in junior col- 
leges, to a "hitch” in the service, it is manifestly impossible either 
to commend or condemn general education. Since general edu- 
cation arose out of a desire to change and thereby improve the edu- 
cational experience of students, it might be expected that evalua- 
tion and research would be the tools for indicating the significance 
of these changes and the need and direction of further changes. 
McGrath’s remarks are entirely consistent with this point of view, 
and McConnell's suggestion that general education may be leading 
"to a complete reconsideration of the nature of the learning 
process" likewise suggests that the role of experimentation and 
evaluation may be less that of passing judgment on general educa- 
tion and more that of providing direction based on scientific evi- 
dence. 


Concern about Evaluation 

There have been numerous projects and programs of evalua- 
tion carried on in reference to general education. The volume 
Evaluation in General Education' includes reports of the evalua- 
tion practices at a representative group of colleges concerned with 
general education. Probably the General College of the University 
of Minnesota leads the field in the number of evaluation studies 
made. The College of the University of Chicago, Michigan State 
College, Stephens College, Antioch College, and many others, also 
have a lengthy and extensive history of concern about evaluation 

'Evaluation in General Education, ed. Paul L. Dressel pubuque, Iowa: Wm. C. 
Blown Co., 1954). 
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and provide evidence that general education has not been lacking 
in sell study Most such studies have had only limited impact on 
colleges other than those engaging m them, many o£ them, m fact, 
have not been generally available 

On a national scale several projects which have dealt with gen- 
eral education problems have made contributions to evaluation 
The Cooperative Study in General Education,'® directed by Dr 
Ralph Tyler and under the sponsorship of the American Council 
on Education, developed numerous instruments useful in assessing 
growth in regard to general education objectives The Evaluation 
Staff of the Eight Year Study of the Progressive Education Associa 
tion," although at the high school level, attempted to evaluate 
objectives identical with many of those accepted as general educa 
tion objectives 

In 1948 Dr George Angell conducted for the American Council 
on Education a survey of evaluation practices and related these to 
general objectives Among the colleges surveyed it was apparent 
that most evaluation was strictly content oriented, but that objec 
tives involving more than knowledge were commonly accepted 
even though student achievement relative to them was not being 
evaluated 

The American Council on Education has been mentioned 
several times as a sponsor of various projects connected with gen 
eral education Having displayed such interest in the problems in 
the past, it was natural that the Council, through its late president. 
Dr George Zook, should sense the need for evaluation activity 
The way was prepared in part, by the fact that in connection with 
the organization of the Educational Testing Service, the Council 
had given up its production and administration services in testing 
and had received a financial grant to assist in the exploration of the 
new directions its work m measurement and evaluation might best 
take The challenge of McGrath, then United States Commissioner 
of Education, and dthers was directly responsible for focusing at 

'•Cooperation m General Education A Final Report o) the Executive Committee 
of the Cooperative Study in General Educatum (Washington American Counal on 
Education 1947) 

“ Eugene R Smith and Ralph W Tyler et al Adventure m American Education, 
Vol in~Appraising and Recording Student Progress (New York Harper & Bros 
iai9\ ^ 
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tention on general education as an area where such evaluation ac- 
tivity was highly desirable The previously noted survey by Dr 
George Angell confirmed that need It was indeed fitting that the 
American Council on Education, through its Committee on 
Measurement and Evaluation, be the means of instituting a project 
concerned with evaluation in reference to general education 

First Steps 

In October 1949 the Director of the Study was asked to initiate 
steps which might lead to an evaluation project involving coopera 
tive activity by a number of colleges A mail survey of forty four 
colleges was conducted to determine interest in a cooperative eval 
uation venture The survey involved three questions— the need for 
such a project, willingness to participate in it, and willingness to 
contribute to it Twenty seven of the administrators responding 
indicated a readiness for evaluation activity, but some were indif 
ferent and a few were politely antagonistic Reasons for the lack 
of interest were in themselves informative One institution, with 
some reason, felt that it was so far ahead of others in evaluation 
that It would derive no benefit from a cooperative project One or 
two others felt that evaluation must be a purely local activity 
Several who indicated a desire for evaluative activity felt that their 
own programs were too new to be ready for evaluation From the 
favorable responses a group of eighteen colleges was selected and 
invited to send two representatives each to an initial meeting held 
at Pittsburgh in December 1949 Those in attendance were ad 
ministrative and evaluation officers From a two day discussion 
agreement emerged that a cooperative project was desirable, that 
It should be planned and executed by intercollege faculty commit- 
tees, and that six committees should be formed dealing with six 
objectives found to be accepted by most, although not all, of the 
colleges present 

As the major argument for the cooperation of teaching faculty, 
the group insisted that too much of what had already been done 
and said about general education had failed to reach or aifect 
teaching faculty Similarly, evaluation activity would be very un 
likely to affect faculty members unless they were directly involved 
It was also recognized that a project involving cooperative effort 
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might be slower moving and perhaps less prolific m the develop- 
ment o£ instruments and accumulation of evidence than a full time 
central research staff, but the majority of those present were will- 
mg to sacrifice such accomplishment to the end of rapport ivith and 
involvement of teaching faculty Finally, there was the conviction 
that teachers have a unique and vital contribution to make to any 
evaluation of instructional programs 

The SIX committees specified were to be related to the following 
SIX objectives ** 


1 ‘ To partiapate actively as an jnfonned and responsible citizen in 
solving the social, economic, and political problems of one s com 
munity, State, and nation" (Committee on Social Science Ob 
jectives) 

2 ‘ To understand the common phenomena in one’s physical en 
vironment, to apply habits of saentific thought to both personal 
and CIVIC problems, and to appreaate the imphcauons of saentific 
discovenes for human welfare" (Comimttee on Science Objec 
lives) 

3 'To understand the ideas of others and to express one's own 
effectively' (Committee on Communications Objectives) 

4 ‘To attain a satisfactory emotional and social adjustment" (Com 
mittee on Attitudes, Values and Penonal Adjustment) 

5 ' To understand and enjoy literature, art, music, and other cul 
tural activities as expressions of personal and social experience, 
and to participate to some extent in some form of creative activity ’ 
(Committee on Humanities Objectives) 

6 ‘ To acquire and use the skills and habits involved in critical and 
constructive thinking ’ (Comimttee on Critical Thinking) 


A survey of the eighteen colleges which sent representatives to 
the Pittsburgh meeting had requested that the objectives from 
this (or another similar) list appropriate to the local program be 
checked The preceding six objectives were the ones with the 
highest frequency The four immediately related to subject matter 
(numbers 1, 2, 3, and 5) were almost unanimously selected, the 
remaining two were somewhat less popular Although these objec 
lives were taken as the basis for deciding that there should be six 
areas of attack and six intetcollege committees, by general agree- 
ment It was left to the respective area committees to determine 
just what they should do 


“ Higher Education for Amertcan Democracy 1, 50-£8 
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Following the Pittsburgh meeting the formation of the Coopera- 
tive Study was authorized and sixteen of the eighteen colleges 
represented at the Pittsburgh session joined the project. Three 
others joined within a few months, and this group of nineteen con- 
tinued as participants for the entire period of the Study. 

Each of the nineteen colleges was asked to decide with which 
of the six objectives or areas it wished to be involved. The decision 
was based on the interest and availability of staff members, on the 
stage of development of the various courses, and the financial 
burden imposed by travel and other expenses attendant upon 
participation in a committee. The number chosen by the various 
colleges ranged from three to all six, and varied somewhat over the 
period of the project.^* With very few exceptions, the faculty 
members chosen were truly interested and active in the venture 
add demonstrated thereby that considerable care had been used in 
selection. At the request of the college administrators, the Director 
of the project had visited each campus prior to the formation of 
the intercollege committees. This was done so that the project 
could be presented directly to the faculty, and so that the project 
Director might have an opportunity to become acquainted with 
programs and with individual faculty members. These visits also 
enabled him to participate in the selection of intercollege com- 
mittee representatives. 

Two things were clear (to the Director, at least) from these 
early visits: (1) that teaching faculty were generally interested but 
somewhat less concerned about evaluation than were the admin- 
istrators,* and (2) that general education objectives and programs 
at the practicing level do not correspond well with the descrip- 
tions of them given by administrators or individual devotees of 
general education. 

In fact, some faculty members (a minority) evidenced very 
definite disagreement with the implication that evaluation was 
necessary. One comment made several times in different places 
was to the effect that evaluation results are valid only when 
they confirm the considered expert judgments of those who set 

** Although Harvard University was ably lepresented on the Committee on Meas- 
urement and Evaluation by Dr. David Owen, it did not tend representatives to any 
oi the intercoUege committee meetings. 
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up the program A larger group o£ general education instructors 
was inclined to be somewhat dubious o£ objectives other than 
knowledge o£ content Pushed lor a definite stand many would 
admit to a beliel that most general education objectives such as 
critical thinking depend upon knowledge and lollow automati 
callyoutoEit 

Thus, in an atmosphere charged with some enthusiasm but 
containing reasonable proportions of doubt and skepticism and 
a dash of antagonism, the activity of the Cooperative Study of 
Evaluation in General Education began In general, by the pro- 
clivity of the Director and the committee members, the project 
was inclined to de emphasize philosophical discussions of ob 
jectives in favor of operational and behavioral descriptions, to 
evade arguments about course content and program organiza 
tion m favor of the development of instruments or of teaching 
techniques usable m any program, and to substitute work for 
discussion The extent to which such a course was held to and 
was justified by results is recorded m the following chapters 

The General Plan of Operation 
The actual operations of the Study involved four distinct types 
of activity on the part of the intercollege committee members 
committee meetings during the school year, summer workshops, 
individual and subcommittee activities, and local campus activity 
Each of these phases of activity contributed heavily, although 
somewhat differently, to the program 
CommiUee meetings — Initiaf committee meetings were held in 
March and April, 1950, in Cleveland, Ohio At these sessions each 
committee met for three days and attempted to map out the prob 
lems to be attacked and the procedures to be used in so doing 
This ivas accomplished with varying facility and success by the 
SIX committees The Director of the project served as chairman 
of each committee session, attempting to permit the committee 
the greatest freedom of exchange of ideas, but also encouraging a 
synthesis of ideas and the definition of common but nonetheless 
major significant issues for attadt 

Subsequent midyear committee meetings, held in Cincinnati 
and East Lansing, filled somewhat different roles Review and re 
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consideration of activities and plans laid in the summer work- 
shops, study of the individual contributions of committee mem- 
bers and colleges, and formulation of further and more specific 
commitments for further contributions became the major activi- 
ties of these later sessions. 

Summer workshops.^Summer workshops of from one to two 
weeks in length were held in the summers of 1950, 1951, and 
1952. The first and third were held in August, the second in 
June. These sessions provided the committees with a period for 
concentrated work during which the materials contributed by 
individuals during the year were criticized and assembled into 
an instrument, report, or plan. Full committee sessions, subcom- 
mittee sessions, and individual projects coupled tvith occasional 
full workshop sessions to discuss matters of common concern 
resulted in participants working from early morning until late at 
night. A twelve-hour day was the rule rather than the exception. 

Every attempt was made to supply consultants, reference mate- 
rials, and the like which were requested by workshop participants. 
Such expediting steps coupled with the enthusiasm and assiduous 
efforts of the participants made the workshops the major item in 
forwarding the work of the Study, 

Individual and subcommittee activity.— Occasions arose fre- 
quently when further development of a plan of a committee pro- 
ject required extensive work on the part of a few individuals. 
Where this involved only a few days up to a week, it was possible 
for the Study by paying expenses— travel and maintenance— to 
arrange for the persons involved to come to the central office for 
this work. In one case arrangements were made for a small 
group to work together for several days on the campus of another 
of the cooperating institutions, but more commonly such small 
working groups assembled at the central office. The time and 
effort thus contributed played a significant role in moving along 
various phases of the Study. Had the Study budget permitted it, 
much more of this type of activity would have been carried on. 

Local campus organization.— It was quite apparent at the begin- 
ning of the Study that the central staff could not satisfactorily 
function in reference to the many members of committees on 
each of the given campuses without some coordination being 
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provided at that level. At the start o£ the 'Study, thm, each 
cooperating college was ashed to name a local liaison officer. In 
some cases these liaison officers were active participants in one 
or more committees; in others their role was that of maintaining 
a general awareness of the entire scope of activities of the Study 
and meeting frequently with individuals and with the group 
of people working on various phases of the Study projMts. These 
liaison people tended to be persons having administrative respon- 
sibilities for the general education program, although occasionally 
they were chairmen of evaluation or research committees or the 


evaluation officer himself. 

Many of the colleges further organized local faculty committees 
related to each of the intercollege committees in which the college 
participated Usually, although not always, the chairman of this 
local committee was the designated representative to the inter- 
college committee. In this way the person serving on the inter- 
college committee reported viewpoints from a definite group on 
his campus, and was also in a position to secure opinions and 
contributions from his committee in carrying out the work of the 
Study as planned by the intercollege committee. The significance 
of such local organization was pointed up by the difficulties 
found by those intercollege committee members having no such 
committee in obtaining a hearing or in arousing interest upon 
their return home after each of the meetings. 

At all stages of the Study the cooperating colleges had complete 
autonomy, not only m the committees sefectcJ, but also as to the 
extent to which they contributed to any phase of the work of a 
committee. For example, the Communications Committee was 
concerned with all the communication skills, but on some cam- 
puses speech is not included in the freshman course. In such cases 
no contribution could readily he made and none was expected 
Deviation from the planned program of a committee was not 
necessarily in the direction of abstention from certain activities 
In a number of cases individuals developed an idea related to the 
committee planning or proposed some additional step which 
they carried out independently of the committee. The number 
of such individual projects was not as large as originally expected 
simply because specific commitments of individuals to thfeom 
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mittee program when added to their full teaching load left 
little time for additional endeavors. 

Summary 

The general education movement originated as a reaction 
against overspecialization and compartmentalization. Objectives 
have played a prominent role in the development of general edu- 
cation and have provided some degree of commonality among 
very diverse manifestations of the movement. Three reasonably 
distinctive emphases have been noted: 

1. Content orientation wherein the primary concern is with the 
importance of the materials selected and the task of education is 
to expose students to these materials. 

2. Intellectual orientation wherein attention focuses on certain 
intellectual objectives, and subject matter is selected in part 
because of its relevance to the attainment of these objectives. 

3. Student orientation wherein the primary value is the total 
development of each individual and wherein experiences are 
developed or selected with this end in mind. 

These variations in viewpoint and associated variations in ma- 
terials and methods suggested the need for experimentation and 
evaluation to determine whether one approach is better than 
another. Recognition of this challenge resulted in the formation 
of the project here reported upon. 

The six intercollege committees working on six objectives 
accepted by all or most of the colleges undertook to define the 
objectives more carefully, develop evaluation techniques, and col- 
lect evidence on student achievement. The major portion of this 
volume is devoted to a report of the procedures and the results 
of this committee work, but a brief examination of the concept of 
evaluation which in part controlled, and in part, developed out of 
the work of the committees is a desirable prologue. 
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T he initial chapter briefly examined the origins of general 
education and indicated why both the proponents and op- 
ponents of general education might feel that general educa 
tion needs to justify itself It was suggested that the task thereby 
implied IS an extremely difficult one As the committees of teachers 


representing the various general education courses and programs 
entered upon their deliberations, it also became apparent that, 
to them, an attempt to evaluate general education seemed alto 
gether premature Evaluation to many of these teachers carried an 
implication of finality — a judgment of goodness or badness which 
they, being dedicated to general education goals, were uninter- 
ested in and unwilling to make 


There was, in most cases, no hesitation about admitting that 
their present efforts at general education might be inadequate and 
that some indications of adequacy or inadequacy were needed 
There was, however, less interest in such evidence than in the 
inferring from it and from the attempts to collect it, what steps 
should be taken to improve existing programs Evaluation regarded 
as a judgment of the worth of general education was also 
deemed premature in that most of the committee members 
felt that the full significance of general education objectives 
was not >et realized and that certainly the means for appraisal 
of their achievement was not yet available Quite obviously 
some individuals Mewed evaluation as a personal threat in that 
It might reveal course or personal inadequacies which would 
be used against them Assurance had to be given that intercourse 
and intercoilege comparisons would be made at the discretion 
of the participants and would be kept confidential if revealed 
to Uie staff of tlic Study. The use of the preposition "in” rather 
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than “o£” in the title o£ the Study carried significant and 
meaningful overtones for most of the participants. 

Having assimilated and accepted the point of view implied by 
the preposition "in,” the committees readily turned their atten- 
tion to the development of an evaluation program. Naturally, 
most of the participants were neophytes in evaluation and were 
concerned with achieving an understanding of the nature of 
evaluation and the possible purposes which the process might 
serve. A review of some of the thinking in this regard will make 
clearer the concept of evaluation finally attained and afford a 
basis for appreciation of the efforts reported in later chapters. 

Possible Purposes of Evaluation 

The following possible purposes of evaluative activity were 
identified in the early committee deliberations: 

1. Clarification and possible redefinition of the objectives of general 
education. 

2. Development of more adequate and reliable means of measure* 
ment. 

3. Appraisal of the development of students. 

4. Adaptation of courses and programs to the individual student. 

5. Motivation of student learning dirough continual selFevaluation. 

6. Improvement of instruction. 

The clarification of the objectives of general education might 
appear to be an inappropriate purpose to associate with evalua- 
tion since evaluation is presumed to begin ivith a previously deter- 
mined set of objectives. However, such an objective as "attain- 
ment of a satisfactory emotional adjustment” is so unspecific that 
as one undertakes to define it and to evaluate student attainments 
in regard to it, the very attempt brings greater insight into the 
objective and modifies some of the preconceptions about it. 

The second of these purposes is really only a step in evaluation 
or a means to that end, but it is particularly relevant to general 
education because it is with respect to objectives descriptive of 
behavioral outcomes other than knowledge that the most difficult 
measurement problems arise. Unquestionably, the lack of ade- 
quate and reliable techniques of measurement has impeded greatly 
the placing of suitable emphasis on such go.'ils. In part, this may 
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be a result of the common pattern that things not tested for are 
apt to be Ignored, in part, it may be that our inability to measure 
reflects out lack of understanding of just what it is that we want 
The third purpose— appraisal of the development of students— 
is one often talked about in education but seldom dealt with 
adequately The question of how and how much students change 
as a result of their educational experiences requires an extensive 
program of pre and posttesting over a period of time such as 
one, two, four, or more years Such appraisals of progress are 
diBicult because they involve very careful experimental designs 
if extraneous factors are to be so controlled that conclusions can 
be drawn They also involve technical problems in the proper 
interpretations of test score gains 
The fourth purpose that of adaptation to the individual student 
at the simplest level, was seen to involve the pre testing of 
students tor sectioning or tor appropriate placement m courses 
At a more complex level, it may involve assignment of students 
to special services or the development of individualized courses 
and assignments 

The motivation of student learning through self evaluation is 
a purpose based upon one of the accepted principles of learning 
As objectives become understandable and important to students 
and as information regarding achievement is available to them, 
an increased desire for further improvement results from more 
precise goals and from the satisfaction of realizing that some 
improsement has taken place Self evaluation is a potent learning 
incentive and a procedure too seldom exploited The lack of 
simple yet reliable and valid evaluative evidence with regard to 
certain objectives and the tendency to view evaluation as separate 
from torching may account for much of this neglect 
The improvement of instruction was seen by the committees 
as a major purpose of evaluation This purpose disturbs some 
teachers for it seems to imply that instruction is poor, whereas 
the implication need be only that instruction may be improved 
through study of the instructional process and through determina 
tion of the effects of it on students It should not be implausible 
Uiat instruction— .particularly that aimed at certain previously 
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ignored general education objectives— is less than perfect and that 
therefore it might be improved by conscientious teachers who 
have developed new insights into what they are doing. Accepting 
this view, evaluation becomes a technique highly significant in 
modifying and improving instructional practice through relating 
it more directly to previously stated outcomes. 

All of these six purposes at one time or another were considered 
by all committees and some contribution was made to all purposes. 
However, the initial emphasis in every case was placed on the first 
two purposes: selection and clarification of one or more objectives 
and development of evaluation procedures or devices. The third 
purpose, evaluation of student development, was regarded as the 
next logical step following the first two, and the fourth and fifth 
purposes were touched upon only incidentally. The sixth purpose, 
improvement of instruction, ultimately came to be regarded as 
the most important of all to the committee members who as 
teachers were vitally concerned with that process. The gradual 
move toward this emphasis will be apparent in the activities of the 
four subject-related committees. 

Characteristics of the Evaluation Process 
Evaluation, as contrasted with measurement, embraces a wider 
range of technique and evidence. Evidence of increased participa- 
tion in cultural activity can be accumulated on a simple counting 
basis by recording the attendance of students at concerts, lectures, 
and art exhibits; evidence of increased ability to judge critically 
such activity may require essays, interviewing, or simply subjective 
appraisals by teachers based on class discussions. The primary 
concern is that the best— the most valid— possible evidence be 
collected, and the techniques of evaluation are selected to this end. 
Certainly the restriction to objective tests— a common complaint 
of teachers— is unnecessary and is by no means characteristic of 
the best-conceived evaluation programs. Early in the Study the 
Director suggested a series of questions to be faced in entering 
into an evaluation of general education: 

1. What does this objective mran in terms of definite and concrete 
action, feeling, or behavior on the part of the individual? Faced 
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ivith a RTOup of individuals of unknown educational bai±ground, 
what ivould distinguish between those who have and those who 
has e not attained the objective? 

2 What are some of the specific situauons, problems, or expenences 
svhich confront individuals and m which the type of behavior de 
scribed by this objective is necessary or desirable? What is the range 
of situations in terms of type, complexity, involvement of other 
factors, etc , m which this objective is involved? 

3 At exactly svhat point, or to what elements, in the problem or situa 
tion IS the described behavior applicable? 

4 What other types of behavior seem to be involved in many, or 
most, situations in which the desired behavior is applicable? 

5 Is the desired behavior sufficiently distinct to be noted separately, or 
must It be noted only as it occurs m conjunction with some more 
involved, and perhaps more meaningful, behavior pattern? 

6 Does this conjunction of behaviors and possible inability to 
separate the particular one m question suggest a redefinition of the 
original objective, or a grouping together of several related ob 
jecuves? 

7 Should this objective be related to oihen, or considered simul 
laneously with others in evaluation on the basis of logic, reality, 
or convenience, even though separation is possible? 

8 Is the desired behavior a totality of more speafic behaviors which 
may not be closely related to one another? 

9 Is It possible to place before students, or to place students in, the 
real or true lo-life situations which have been considered? If not, 
what modifications must be made, and on what basis? 

10 Is It desirable to construct a less realistic situation in which achieve 
ment with respect to this particular objective is more easily noted 
and IS more clearly separated from other objectives and from 
random elements? 

11 What kind of record is to be kept of the student s behavior in the 
situation? Hovvr is the record to be analyzed? What is to be done 
with the results? 

12 Of those individuals who are observed to handle adequately a situa 
uon supposedly involving a ceruin behavior (o) Do all show that 
behavior? (6) Do they show it m varying wajs and amounts? (c) In 
what ways do they difier? (d) Do some show complete absence, or 
almost complete absence, of the behavior? 

13 If presence of the desired behavior is not found to be necessary for 
success in the total situation, what are the alternative patterns of 
behavior which succeed, and to what knowledge, experience, inter 
ests aiiitudes, etc., are tlicsc due? 

H Is tlie desired behavior, even though not necessary, deemed desir- 
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able on. aesthetic, ethical, or other grounds? Another way to 
ask this IS Is a certain process, behavior, or activity considered 
good in Itself, even though not necessary for satisfactory handling 
of situations in which individuals find themselves? 

15 What are the specific classroom experiences— readings, demonstra 
tions, discussions, activities, or lectures— which are thought to con 
tribute to the growth of the individual with regard to the objective? 

16 Do the classroom experiences add measurably to the growth of the 
individual as compared with other individuals who have not had 
these particular experiences? If not, js this due to lack of, faulty, 
or inadequate classroom experiences, or is it simply that growA 
with regard to the objective is so slow and complex as to be hardly 
noticeable? 

17 Do the analysis of the objective and the discussion of situations 
involving It suggest any additional activities which might be carried 
on in the classroom, or any different way of teaching? To what 
extent are the evaluation exercises also usable as learning devices 
or instructional techniques? 

18 Do the clanfication of the objective and the accumulation of evi 
dence of growth with regard to it provide any basis for greater 
student motivation? 

19 What relations exist between growth with regard to this objective 
and such factors as (a) intelligence, (5) status at begmning of a 
course, (c) grading of the students? 

20 What IS the optimum growth which may be expected for various 
kinds of students with regard to this objective * 

These questions were not used point by point in the defibera 
tions of any of the committees nor were they intended to be so 
used The scope of the questions does fairly reflect the many 
issues to be faced in careful study of an objective, and the questions 
demonstrate also that no artificial limitations are to be placed 
on the nature of evidence solicited nor on the ways of obtaining 
it Finally, they indicate that evaluation and instruction can very 
profitably be thought of as inseparable 
Approached in this way, it was apparent that evaluation must ^ 
be a cooperative enterprise— cooperative in many ways for many 
reasons The more complex behavioral outcomes require coopera 
tion between teachers and evaluators in developing procedures and 

'Paul L Dressel Evaluation Procedures for General Educauon Objectives, The 
Educational Record, April 1950 pp 97-122 
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in collecting evidence— cooperation of administrators, personnel 
workers, and of the students themselves-if time is to be found, if a 
broad range of evidence is to be collected and if students are to be 
properly motivated Student motivation through cooperation is 
particularly important Evaluation -which has grading as its pur 
pose provides an immediate motivation although one conducive to 
fallacious results in that the answers of students frequently repre 
sent an attempt to guess the intent of the teacher, rather than an 
honest statement of their own individual viewpoints Evaluation 
which has no such competitive motivation may be met with in 
difference by students unless the purposes and the relation to the 
course or programs are made clear Similarly, evaluation done by 
evaluators without the full cooperation of teachers is apt to be mis 
undentood and even rejected Research involving human beings 
cannot be a cold, calculated procedure disregarding the emotions 
of those involved 

The preceding questions imply that evaluation is concerned 
with tlie utility and utilization of results A purely research onen 
tation may be more efficient m obtainmg results but— for reasons 
just noted-it may be ineffective insofar as acceptance and resultant 
change are concerned Some lag between research resulu and prac 
tice IS to be expected but the gap presently existing between psy 
chological investigations and much of instructional practice is an 
undesirable state of affairs Evaluation seeks to remedy this situa 
tion 

The complexity of some of the questions suggests that the use of 
evaluation in improvement of the educational process cannot be 
a single shot proposition but rather it must be an ongoing process 
in which evaluation and instruction interact and continually 
modify each other So regarded, evaluation is a state of mmd as 
much as it IS a process The instructor who critically regards his 
practices eacli day, who seeks to find in his students evidence of 
the effect of his instruction and of their own endeavors in terms of 
progress toward accepted educational goals will find that evalu 
atue-ccntered thinking is a continuing rewarding and integral 
part of his teaching On the contrary, evaluation which is periodic 
or spasmodic is usually not well conceived and has very little effect 
other than the development of a distaste m both teacher and stu 
dents for such ventures. 



RATIONALE OF THE STUDY 25 

Attitudes toward the Role of Evaluation 
IN General Education 

Ruth Eckert* has provided a stimulating and provocative state- 
ment regarding the status and the possible role of evaluation in 
respect to general education. In doing so, she has outlined also 
some of the major issues wherein experimentation or evaluation is 
needed. These are classified broadly as follows: (1) research on the 
aims of general education; (2) studies of students who take general 
education courses; (3) studies of different types of general educa- 
tion programs; (4) studies of instructional methods used in gen- 
eral education; (5) studies of out-of-class experiences in their rela- 
tion to general education; (6) studies related to the staffing of gen- 
eral education; and (7) studies related to methods of evaluating 
outcomes of general education. The scope of this list— for many 
specifics were listed under each of these headings— suggests how 
little we know about educational processes. Many of the points of 
this list correspond to and confirm the possible purposes of evalua- 
tion identified by the committees of the Study. 

The extent of these recommended studies suggests a major, even 
a central, role for evaluation in general education and one which 
has not been much in evidence in practice. Although not a subject 
of formal deliberation, the question of why evaluation has not 
played such a role was discussed informally and repeatedly with 
committee members, faculty members, and administrators. Lack of 
facilities, expense, and lack of trained personnel were frequent 
complaints, but actually appear less important than the disinterest 
and even antagonism of many teachers toward evaluation. The role 
of evaluation in general education is likely to be whatever teachers 
and administrators assign to it. Even in cases where much in the 
way of evaluation has been done, there may be little apparent 
effect in the general education program unless the results are 
understood and accepted by the faculty. 

There appear to be three distinctive attitudes toward evaluation 
which are identifiable in teachers. One of these completely rejects 
evaluation other than the judgment of the teachers themselves. Dis- 
cussion with those holding this viewpoint reveals that they view 

*Ruth Eckert, "Evaluation in General Education,” in The FifiyTirst Yearbook 
of the National Society far the Study of Education, 19S2, Part 2— General Education 
(Chicago: University of Chicago Press, 1952), pp. 250-78. 
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many of the outcomes of instruction as intangible in nature and 
assessable only by the instructor as he is in close contact •with a 
student over a period of time These outcomes may differ mark 
ediy-these teachers say— from instructor to instructor and are not 
readily communicable to others Certainly these intangible ob 
jectives are not amenable to evaluation processes as viewed by ad 
herents of this viewpoint Rejection of evaluation is apparently 
quite common among humanities teachers but is not limited to 
them Subjectively it seems that a majority of this group are also 
strongly content oriented m the actual conduct of their courses 
They are inclined to favor essay testing but may use some objective 
achievement tests which are more often than not largely factual in 
nature and based on recall more than on understanding An ex 
treme reaction from this group is exemplified by the actual al 
though paraphrased comment that evaluation is a waste of time, 
for u either confirms the judgment of those who developed the 
course or the evaluation itself is in error 
A second attitude toward evaluation found among teachers 
recognizes the need for it, but feels that the exigencies of teaching 
make it impossible to do much about results The size of classes, 
the teaching load and the amount of material to be covered are 
cited as factors which force the teacher into the use of formal and 
traditional instructional procedures which— they may admit— prob 
ably are not particularly conducive to student development other 
than m factual knowledge Many teachers will add to this that the 
low ability and poor attitude of many students admitted to college 
nowadays make unrealistic any attempt to develop intellectual 
skills or to use group dynamics procedures which might be more 
conducive to development of social understandings attitudes, and 
values Most of the teachers who feel this way must be regarded as 
either content oriented or intellectually oriented, but the classroom 
practices of even the latter group commonly regress to the content 
orientation Such individuals frequently exhibit interest in evalua 
tion procedures and results and they are usually much interested 
in extending the range of their own testing to include more of 
understanding and application as an antidote to overemphasis of 
rote recall They are not ordinarily much concerned with evalua 
tion of broad general education goals transcending their own 
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courses, nor do they, with the many pressures upon them, see much 
opportunity for adaptation of their courses to individual needs. In 
many cases their generally favorable attitude toward evaluation can 
be utilized in initiating evaluation activity which will, as it be* 
comes productive of neiv insights, result in replacing the somewhat 
dubious and pessimistic view regarding the utility of evaluative 
practices by a positive and even enthusiastic endorsement. Perhaps 
because of their generally opcn*minded and scientific attitude, 
science and social science teachers are particularly likely to be 
found in this second group. 

There are some general education faculty members, and the 
number is increasing, who have had formal measurement and 
evaluation training or who have had sufficient practical experience 
in educational research to be reasonably facile in planning and 
executing an evaluation project. The Study ivas extremely fortu* 
nate in finding in each of its committees a few individuals who had 
carried out significant studies in connection with their own courses 
or in relation to the total educational program. There would seem 
to be~judging from the experiences in working with such indi- 
viduals— a positive but not perfect correlation betiveen active inter- 
est in the broader problems of evaluation and concern for intro- 
duction of a greater degree of student orientation into the general 
education program. More such individuals must be identified or 
developed, ior in them Ties the hope lor trv}y signiBcant develop- 
ments in general education. 

The existence of some of these reactions to evaluation suggests 
that general education faculties are not, to a man, breathlessly 
awaiting the opportunity to join in an extensive evaluation of gen- 
eral education or in a critical restudy of their own courses. How- 
ever, the actual opposition is probably less than it at first appears to 
be, for those most opposed to evaluation are also the ones who are 
likely to express themselves strongly on the matter. The vocal 
antagonism of such a group, especially since it has the support of 
a long tradition of independence on the part of individual college 
teachers in their classroom procedures, is a major difficulty to over- 
come in beginning any basic evaluation or self-study venture. De- 
spite such opposition, there is, as indicated by experiences in the 
Study, a sufficient number of persons already evaluation-minded 
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so that, assuming administrative support, some action can be initi 

aied 

With the committees in our Study containing many evaluation 
minded individuals it might be expected that the luW range o£ 
ma3or issues suggested by Ruth Eckert would be attacked, but only 
in a limited sense was this true At the present moment in the de- 
velopment of general education most teachers are concerned with 
evaluation as it relates to their immediate responsibility in teach 
ing a course a state of mind which does not quite appear to co- 
incide with that of professional researchers and evaluators Hence, 
the question may be raised as to just what teachers do want from 
evaluation In a sense the later chapters of this report provide the 
ansiver because the committees of the Study were largely self 
directed they made their own plans and they attempted to carry 
them out The lack of single mindedness in some of the committees 
betrays the presence of differences in viets point about education 
and about the role of evaluation There are present, nevertheless, 
certain evidences of common desires and hopes as to what evalua 
tion may do We shall attempt to synthesite these into a point of 
view about evaluation which may be a reasonable picture of what 
general education teachers desire It is certainly a description of 
the most pervasive point of view about evaluation represented in 
the Study 

A Point of View about Evaluation 
The student orientation approach to general education starts 
with the concern that as the result of general education experi 
ence, the individual be different — not just in knowing more, but 
different in the way he thinks believes and behaves This concern 
that education make a real difference m people is not a crass utili 
tarian view but one which arises out of the commonly observed 
tendency of individuals to compartmentalize the various areas of 
behavior either because they do not see or are unwilling to see any 
connection College graduates— particularly the specialists— fre 
quently do not apply in their activities as citizens the knowledge 
and skills learned in the classroom because little relationship is 
seen between the mundane problems of living and the cloistered 
academic situation In an attempt to develop improved programs 
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which will bridge this chasm between theory and practice, it is 
necessary to seek for every imaginable type of evidence about stu* 
dent backgrounds, student reactions, and student performances. 
Even the reactions, interactions, and teaching practices of the 
teaching staff are fit subjects for study and change. This broad 
conception of the possible significance of evaluation may be ac- 
ceptable to many teachers, but it is just too remote from their im- 
mediate problems to arouse much enthusiasm. 

The instructor is concerned with instruction, and evaluation 
must make direct contribution to this if it is to be of interest to 
him. Typically he has some objectives but is not quite certain what 
they mean. He is interested in an operational approach to the 
definition of these objectives, an approach which makes it possible 
to detect the presence or absence of the implied abilities or char- 
acteristics in his students. He realizes that because objectives are 
not clear, they have commonly played little part in the selection of 
course materials and experiences which are likely, therefore, to 
have been selected on the grounds of personal judgment as to their 
intrinsic worth. Similarly, his techniques of instruction are com- 
monly highly traditional and bear little relation to the desired 
outcomes. His hope is that evaluation can become the means for 
drawing together into an integrated whole these elements of the 
curriculum ^vhich too often are fortuitously determined rather 
chan sysCemacfcahy pJanneef. 

As objectives are clarified, it often becomes equally clear that 
certain objectives may have been slighted or even ignored entirely 
in the course as offered. The efforts to develop tasks or to devise 
means of observation which will give information about the ob- 
jectives as revealed by student behavior may suggest netv experi- 
ences, ne^v materials, and new tvays of presenting them ^vhich 
relate directly to the objectives. The historical cases in science sug- 
gested and developed by Dr. Conant represent such an attempt to 
select science material which particularly well exemplifies what 
science is and how scientists operate. The attempt of the Commit- 
tee on Evaluation of Objectives in Science to evaluate student 
ability in the reading of current science articles led to an analysis 
of what is involved in such reading, to the selection of appropriate 
readings, and to the specification of what reactions might be ap* 
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propriate to each selection The resulting materials were immedi- 
ately utilized by many of the committee members for instructional 
purposes 

Methods necessarily become important as emphasis is placed on 
behavioral objectives The social science teacher who is concerned 


with good citizenship as an outcome and who accepts some respon 
sibility for it beyond basic knowledge will wish his classroom to 
provide some experience in democratic participaton He may also 
become concerned that students have a number of relevant experi 
ences because he feels that these experiences are more conducive to 
growth with regard to the objectives of the course than were his 
former practices He has already made a subjective evaluation jus 
tifying this change, he may wish a more objective evaluation which 
will test the implied hypothesis In the past and in many general 
education courses at present the matter of method— except for such 
decisions as whether there shall be a laboratory-is left to the mdi 
vidual teacher Any attempt to change this may seem to be a move 
m the direction of standardizing the individual teacher, in a 
limited sense, it is In passing u may be noted that certain kinds 
of standardization are already m evidence, since classes are sched- 
uled at certain hours and, for multiple section courses, common 
materials o£ instruction are usually speciBed Unless each general 
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thusiasm and the student response to it. What a teacher does is 
probably less important than the frame of mind "vvith which it is 
done, for the reaction of students is as much to the intent as to the 
acts of the teacher. The preceding comments lead to the conclu- 
sion that no simple or universal answers about the worth of spe- 
cific materials or specific methods are possible. Effective teachers 
will probably vary both in their materials and their techniques 
with different classes and possibly with different students. Since, 
to date, the extensive attempts to evaluate teaching effectiveness 
have not had any significant results, it is doubtful that research on 
methods of instruction in general education will soon present 
valid answers. 

However, evaluation does provide some suggestions about 
method, suggestions which may not be scientifically validated but 
which nevertheless have a degree of logical validity. For example, 
the attempt mentioned above to evaluate the ability to read cur- 
rent science materials resulted in a finding that textbook materials 
and popular scientific articles are quite different in nature. If it is 
really desired that students learn to read the latter, the introduc- 
tion of a series of such experiences into the course seems appropri- 
ate. Particularly is this true if the habit of reading such materials 
is to be developed. The use of such materials does not have behind 
it the weight of tradition, but in respect to the'purposes stated, it 
has as much logical validity as exists in the use of a textbook. 
Hence, there is no reason to ask nor to expect that a teacher await 
the results of an extensive research study before modifying his 
teaching to include such materials. 

A related point is that much of general education instruction 
has been so traditional that few distinctive and well-organized 
instructional approaches can be found for comparative evaluation. 
The slight modifications found are like fresh frosting on a stale 
cake; the appearance is improved but the essential character is 
unchanged. Much more needs to be done in the devising of new 
materials and new techniques with general education objectives in 
mind. The various committees of the Study found that their at- 
tempts to clarify objectives and to develop ways to evaluate them 
led to the development of problems and materials which— almost 
without exception— ivere appropriate and better means for in- 
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structing or for giving students practice and experience conducive 
to growth with regard to the objectives The developing pattern of 
activity in each of the six committees of the Study over their three 
year period of operation exhibits an increasing concern with the 
instructional implications of their work on evaluatory procedures 
This concern even superseded the original, rather strong, interest 
in the evidence accruing from pre and post testing with the instru 
ments developed At first thought this may appear regrettable but 
it was actually a decision— not entirely a conscious one— of consider- 
able wisdom Preliminary evidence pointed to the fact that rather 
small gains, if any, were present Careful review of the instruments 
and the mode of development provided an assurance of validity 
Study of classroom activity and the reactions of students to whom 
the instruments were administered indicated a paucity of expen 
ence in the classroom directly reliated to the objectives under study 
Since the evaluation activity in itself suggested ways to introduce a 
relevant set of experiences it appeared more important to spend 
some time on these matters than to devote it all to assessing changes 
in students with regard to objectives which had been largely 
Ignored in the classroom 

Evaluation has frequently been defined as the placing of value 
on an educational experience It is equally true that evaluation 
activity may suggest ways of placing more value into the educa 
tional experience In the present state of general education it 
seemed more important to diose involved in this project to empha 
size the latter point of view without, however, entirely neglecting 
the former 
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Intercollege Committee on Social Science Objectives: Wendell 
Bash, Colgate University; Harry D. Berg, Michigan State College; 
James A. Burkhart, Stephens College; Alfred Cope, Syracuse 
University; Golda Crawford, Kansas State College of Agriculture 
and Applied Science; Lynn Curran, Boston University; Paul 
Hanna, University of Florida; Edith M. Huddleston, Educational 
Testing Service; Norbert R. Mahnken, Oklahoma Agricultural 
and Mechanical College; George H. McCune, University of 
Minnesota; Doyle Mikesell, Drake University; H. T. Morse, 
University of Minnesota (Consultant); George Neel, Muskingum 
College; Arthur Reynolds, Colorado State College of Education; 
Benjamin F. Rogers, Florida State University; Verne Sweedlun, 
Kansas State College of Agriculture and Applied Science; Meyer 
Weinberg, Wright Junior College 

A LMOST ALL SOCIAL scicnce courses, regardless of whether they 
/\ are historically oriented, problem-centered, or surveys of cer- 
XJL tain social science disciplines^ postulate some common ob- 
jectives. Teachers in such courses would like their students to be 
able to apply knowledge and social science techniques to current is- 
sues and problems. They would like their students to sense, know, 
and understand interrelationships existing between social science 
areas and other branches of human knowledge. Quite naturally they 
want their students to put into practice in everyday life those 
skills, abilities, ideas, and the knowledge accumulated in the class- 
room. Perhaps their most commonly held hope is that students of 
social science courses ivill be better citizens than individuals who 
have not had such educational experiences. Thus, the development 
of the various traits incident to good or effective citizenship is listed 
more frequently than any other of the objectives of social science 
courses. 
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At Its first meeting the Intercollege Committee on Social Science 
Objectives considered many such objectives, seeking to find some 
common interest The deliberations of this committee during its 
organizational meeting throw into relief a number of issues of 
concern to any group struggling with the problems of preparing 
meaningful educational experience in the social sciences 
The question of course content is a difficult one If general edu 
cation is conceived of as * those phases of nonspecialized and non 
vocational education that should be the common possession, the 
common denominator, so to speak, of educated persons as indi 
viduals and as citizens in a free society,**^ there is some logic in 
seeking to establish common course content Such a goal, however, 
IS contrary to the diversity so necessary m colleges serving a nation 
as large as this one Nevertheless the Social Science Committee 
gave careful consideration to the development of a project to 
determine what items of content should be included in a general 
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Having rejected these possibilities, discussions returned to the 
development of good citizenship A number of studies about the 
development of good citizenship characteristics were noted * None 
of these has satisfactorily defined what good citizenship actually is 
Defined one way, good citizenship consists merely of the practice 
of certain rudimentary mechanisms incident to a democratic gov 
ernment Such things as voting, knowing the provisions of the Con 
stitution, and keeping abreast of current events are exemplary of 
this minimal definition At the other pole lies the conception of 
citizenship as synonymous with the democratic way of life or the 
corpus of Christian ethics Viewed in this latter light, citizenship 
can no more legitimately be claimed as the exclusive province of 
the social sciences than of the natural sciences, the humanities, 
religion, or any other aspect of college life If good citizenship were 
defined narrowly, the only possible measure of achievement would 
be to obtain evidence of how many times graduates voted, went to 
town meetings, contributed blood, or wrote letters to editors Such 
a follow up study as this would involve ways and means obviously 
impossible for the resources and tenure of the Cooperative Study 
Conceived in broader terms, evaluation of good citizenship would 
involve such a wide variety of tests of knowledge, attitudes, values, 
and beliefs that their mere development was clearly beyond the 
hopes of attainment by the Social Science Committee 

Critical Thinking as an Objective 

Citizenship appears to demand one skill above all others Even 
in voting for prospective office holders, the citizen must be able to 
analyze the candidate with respect to the issues of the particular 
election and the demands of the office The good life in a demo 
cratic society also seems to rest fundamentally on one s ability to 
think critically about the problems with which he is confronted 
The essence of the democratic creed is that each person possesses 
potentialities for discovering his own problems and for developing 

’Thomas H and Dons D Reed Etfaluatxon of CiUzenshtp Tratning and Incentive 
in American Colleges and Vmversiltes (New York The Citizenship Clearing House 
19j0) Studies have also been earned on by Citizenship Education Project Teachers 
College Columbia University Cituenship Education Study, Detroit Public Schools 
and Wayne University 
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personally satisfactory and socially acceptable solutions to them, 
so that he has no need to defer completely to the will of an author- 
ity, although he is perfectly willing to make use of expert opinion 
when relevant. 

In view of the importance of critical thinking in citizenship, the 
Intercollege Committee on Social Science Objectives elected to 
develop a project involving this skill. However, numerous impor- 
tant questions were raised. (1) Is critical thinking teachable with- 
out reference to any particular body of knowledge? If so, is critical 
thinking a legitimate objective of social science courses or is it 
rather the responsibility of courses in logic or syllogistic reasoning? 
(2) Is critical thinking merely the intelligent use of knowledge? If 
so, should courses make explicit an objective which is assumed to 
be merely a concomitant of acquisition of knowledge? (3) Granted 
‘ science courses might legitimately concern themselves 
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familiar, some approximations of the effectiveness of education are 
possible and estimates of changes with respect to critical thinking 
can be accumulated. 

Critical thinking frequently begins when the individual be- 
comes aware of an obstacle between himself and some personal 
goal. His effectiveness may be inferred from the rvay in which he 
analyzes this problem and hypothesizes and tests solutions to it. 
Critical thinking in this sense is a highly personalized ability which 
defies precise measurement. However, certain intellectual prob- 
lems of concern to students can be posed to determine how well 
the students can solve them. A test can rarely pose problems pos- 
sessing emotional impact comparable to that of personal problems; 
yet test situations possessing sufficient emotional loading to chal- 
lenge a majority of students to overcome their prejudices, biases, 
and preconceptions have been created. 

The issue of creative thinking was regarded as important but 
outside the province of the committee to resolve. The committee 
was unwilling to take sides as to any differences which might exist 
between the creative act and the critical act. Rather, the committee 
chose to assume that there was room in the college curriculum for 
both creativity and criticism. Further, the committee was willing 
to accept, for purposes of compromise at least, the hunch that 
creativity and criticalness might be merely differing degrees of the 
same essential process. 

Thus far the concept of critical thinking has not been defined 
nor analyzed. During the early deliberations of the Social Science 
Committee, the members talked at some length about a project in 
critical thinking on the assumption that other people were gen- 
erally aware of what was meant. This assumption was proved 
erroneous, for while various definitions were suggested, all proved 
too abstract for evaluation or teaching purposes. The only recourse 
was to define critical thinking as the sum of certain rather specific 
behaviors which could be described and which could be inferred 
from student acts. These abilities which ^vere developed a priori 
by the members of the committee are indicated below. This list 
is the result of modifying earlier lists in terms of behaviors ob- 
served in classes and from study of test items. 
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THE CRITICAL ABILITIES 

To identify central issues —One of the basic skills in critical thinking 
IS the ability to identify the central issue or main theme The thesis 
may be perfectly clear, it may be hidden in a mass of verbiage, or 
It may be unstated Until the student has identified the central 
issue, an analysis of the information cannot proceed on a sound 
basis ^ 
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d) To distinguish between relevant and nonrelevanL—To analyze 
social situations and problems adequately, an individual must be 
able to distinguish between those facts that have a bearing upon 
the solution and those that do not. One should ask, “Does this 
statement define, illustrate, or bear upon the problem?” This 
ability is less complex than the one which follows, because it does 
not require the individual to judge the degree of relevancy, but 
only to sort the aspects of a situation into those which do or 
do not have a bearing upon it. 

e) To distinguish between essential and incidental— Those facts 
which are essential to a given situation are often confused with 
other facts which are present but are not a necessary part of that 
situation. Relevant data are not necessarily essential to an in- 
terpretation and may be of only secondary importance. 

f) To recognize the adequacy of data.— An appreciation of the con- 
nection between adequate facts and a valid conclusion is an es- 
sential ability in critical thinking. A judgment made on the basis 
of fragmentary evidence is likely to be of little value. In dealing 
with social issues, it is particularly important that judgments be 
based upon sufficient irdormation. It is also important to be able 
to detect that significant data have been ignored or omitted. The 
omission may have been unintentional, but often the additional 
evidence has been purposely suppressed in order to strengthen the 
argument advanced. In many cases consideration of neglected 
material will destroy an aigument completely. 

g) To determine whether facts support a generalization.— Tczcts may 
be relevant, essential, and adequate but still not support a gen- 
eralization. Furthermore, poorly selected facts occasionally contra- 
dict and seem to disprove a generalization. Also, in some cases the 
support furnished by one fact is stronger than that furnished by 
another. 

h) To check consistency.— AM arguments must be checked for in- 
ternal consistency. Identification of a major inconsistency may 
invalidate a presentation, and in any case an argument cannot 
be considered as a logical whole when it is based upon contra- 
dictory elements. If an argument withstands the test of internal 
consistency, it still must be submitted to a check for consistency 
with other known data. Having recognized the external con- 
sistency or inconsistency of the argument, one is ready to draw a 
conclusion. 

4. To draw warranted conclusions.— The drawing of a warranted con- 
clusion involves making an inference. An inference is a truth or 
proposition drawn from another which is admitted or supposed to be 
true; a conclusion; a deduction. An individual needs to realize that 
certain facts not explicitly stated may be inferred as true or untrue. 
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It 15 also important to realize the limitations of the inferences which 
can be made from given data Many statements at first glance appear 
plausible, but if the inferences are properly drawn their meanings 
may change 


Validity of the List of Critical Abilities 
Any group of college teachers can prepare without too much 
effort a similar classification which, in the abstract, is quite satis 
factory The difficult task is to prepare a classification which is 
internally logical and related to reality The list developed by the 
Social Science Committee gamed considerable validity from two 
quarters The Cooperative Study Committees on Science, Critical 
Thinking Humanities and Social Science each decided mdepend 
ently to work on objectives involving the concept of critical think 
ing Each committee proceeded quite independently, yet the re 
spectue final lists were markedly parallel This is not a particularly 
startling phenomenon considering that the members of all four 
committees are products and exemplifications of a Baconian system 
of logical thought But the similarity of their conclusions indicates 
that they were each dealing with a particular phenomenon which 
exists at least in ‘Wesiem culture and possesses meaning and value 
for teachers ® 

The second source of validity for the list of specifics rests with 
KIT re cvance to actual observable human behavior After the 
hst had been created, members of the committee were asked to find 
e udence of demonsttation or lack of demonstratton of these ab.h- 
a materials and test 
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which student possession or lack of possession of these behaviors 
might be assessed. In the course of working out such techniques 
the committee became faced with new issues. 

Social Science Coukse Content 

The first of these involved a matter already touched upon— the 
element of course content. Fifteen of the nineteen participating 
institutions were represented on the Social Science Committee. 
Each of these institutions possessed certain social science courses 
which, judging by the titles, differed from each other as much as 
night does from day. Would it be possible for such a heterogeneous 
group of courses to use the same measuring instrument? A study 
was made of the specific content of social science courses from 
seventeen cooperating schools with a vieiv to finding out tvhether 
or not they were as distinctive as their names implied.* Tables 1 
and 2 reveal some of tlie findings of this survey. These data permit 
the generalization that there is a great deal of commonality among 
social science courses— much more than has heretofore been sus- 
pected. 

Issues Involved in Preparation of an Objective 
Test of Critical Thinking 

Closely allied to the matter of course content is the degree to 
which a test of critical thinking should also be a test of knowledge 
in the social science area. This issue brings into relief several points 
of view regarding the nature of critical thinking. There were some 
who believed that whatever test was developed should demand a 
fairly sophisticated level of knowledge of social science content. 
Others could visualize a self-contained test of critical tliinking 
demanding no technical knowledge. The committee compromised 
and decreed that its test should not demand more knowledge in 
the social sciences than a majority of entering college students 
could be reasonably expected to possess. They decided also to find 
out something about the role played by knoivlcdge in critical 
thinking. The test items which were eventually prepared into a 

*This jtudr, conjiderablr expanded, hai been published. Do)Ic Mikcscll. "Sodal 
Science General Education Courses,'* Junior College Journal, January 195t, pp. 26S- 
57. 
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form of a test were screened and all terms which seemed even 
Tcmolely technical iverc used as the basis for a separate vocabulary 
test. The Test of Critical Thinking m Social Science and the Vo- 
cabulary Test were administered to the same students to test the 
h)pothcsis that knowledge oE terminology was essential for critical 
tliinking but not sulTicicnt of itselE to ensure success in critical 
thinking The several studies, to be reported later, thoroughly 
supported die hypothesis Students with low scores on the Vocab- 
ulary Test were unable to do anything with the critical thinking 
test Equally significant, however, was the fact that some students 
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with good vocabulary scores did very poorly on the critical think- 
ing test 

Two other issues had to be faced at this tune The first of these 
involved the limitations of existing testing techniques Many indi- 
viduals contend that ability to do critical thinking cannot be as- 
sessed by means of objective-type test questions They feel that the 
presentation of several responses “gives away much of the show.” 
Their solution to a measurement problem would be to ask students 
to respond orally or in writing to a problem situation. This solu- 
tion, however, has two serious limitations Scoring of students’ 
written or oral responses poses such a formidable task as almost to 
preclude dealing with large numbers of students. The second 


TABLE i 

Topics Most PREQtrENTi.T TAtroHT in Social Science Cotmsss 
IN Seventeen Participatino Colleges* 



* 

■> 

c 



F 

G 

>> 

i 

j 

s 


u 


0 

No OF 

No Or 
Cox,. 
Lxoza 


a 

f> 

e 


» 

/ 


A 

t 

g 

A 

' 




p 






CouBBxa 


























V 

V 



Q 


y 


y 


y 


y 

y 

y 

V 


y 



y 

12 

11 

i Culture oooeept 

V 

V 

y 

y 

V 


6 


y 

y 



V 

y 

y 



y 

y 

J 

y 


y 

y 

y 

y 

? 

22 

12 

12 

12 

4 ForsiAtioa pereoaobt}' 

V 




y 


y 




y 










V 

B 

B 


V 




y 


y 


y 

y 

y 


y 

y 




y 



V 

22 

10 


V 


y 


y 


V 


V 

V 

y 


y 

y 

V 







10 

10 


V 




y 

y 

V 

y 

y 


y 









y 



0 

4 Community 

V 

y 

y 


y 








y 


n 




y 


V 

8 

7 

B Social ^oupf 

V 

y 



y 








y 

y 

y 




y 


V 

7 

0 

3 Aiaenca!} Sons <i3 gffversaest 



y 


y 


y 


y 

y 

y 


y 

y 

y 

y 

y 

y 


y 

y 

37 

15 

2 Katitre of government 


y 



y 

V 

V 

y 

C4 

y 

y 



y 

y 

y 

y 

y 

y 

y 






y 

n 


y 


y 


n 

y 

y 



y 

y 

y 

y 

n 

H 

y 

y 

10 

15 



y 





y 


y 

y 

V 




y 

y 

V 

y 

D 

El 

y 

14 

13 

B Pressure groupJ and propaganifa 

y 

y 



y 


y 


y 


V 




y 


y 

y 

y 


y 

14 

12 



y 

y 






D 

y 

n 



y 

y 

y 

y 

y 

y 

y 

y 

20 

15 

2 American capitalism 


y 

y 


H 




Cl 

y 




Q 

y 

y 

El 

□ 

D 

El 

y 

15 

14 



y 



H 




y 

y 

y 



Cl 

n 


If. 

■: 

M, 

Q 

y 


11 

4 Personal incntne flistnbution 


y 



a 




C4 

y 




y 



■ 

□ 

D 

M 

y 

9 


5 Money, banking and credit 


y 








y 

y 






B 

Ml 

M, 

B 

y 



6 Business cycles 


V 







cl 

y 

y 



y 



y 

Bi 



E|i 



7 Marketing and price 


y 







y 


y 



y 



y 




■1 




V 

y 


y 

y 

V 

n 


y 

V 




V 

V 

y 

y 

y 

y 

y 

B 

19 

15 



y 





y 


V 




y 

y 

y 

y 

y 


y 

y 


1« 

14 

S American values 

V 

y 


y 

y 


n 


y 

n 




y 

y 


y 


y 

y 

Dl 

IS 

12 

4 Comparative systems 

y 

y 



y 

y 


y 

y 

V 




y 

y 


y 

y 



y 

IS 

13 










n 

n 




y 







Cll 



Q Cml liberties 




y 


y 



y 

y 




y 

y 

y 



y 

y 

y 

14 

ll 

7 Agriculture 


y 



V 




H 

Ef 




y 



y 

n 



. 












Cl 





E4 







rii 



0 Population 





V 


e) 



n 




H 




Cl 



y 










n 



y 




Cl 





y 




7 

It Consumer problems 









n 

Cl 



y 





Cl 




« 
















y 







V 

e 

0 

13 Crime 


















D 



y 

5 

5 












Q 







Ml 



V 

5 

5 

IS Natural resources and conservation 





y 


y 


y 


i 




y 







4^ 

^ 


* C<4Qma heaiiagt refer to tbe larLtsUoai ud coortee (B<lKsted ta T«bfo 2 






GENERAL EDUCATION 


TABLE 1 


Clasbotcatioii or 


SoaAi SoiEici! CojmsEs Oitobed bi Setoteen 

PARTICIPATIKQ COLLEGES 



TrvB or Counsss | 



Yean 

Taken 

_ 

Bislory 1 

Nonhistorical | 

Sem 

R*- 

UIBED 


Votid 

1I£l 

intc- 

rated 

’rol> 

km 





A Botlon Umveriity 

*. Humin ReltUons (J jfij 

FuftYeir 

SMond Y«»f .... 

h Fobliul EeoDomic* 

B ColCTte Uoiversity 

« Public Affwi , , Tt, 

d. Autneanldtu in the Modern World 

C, Colorido SUte College of Eduentwn 
«, Contemporary InititutJona and Proh* 
letni in Hutorical Pcripeetire 

D l>rake Unirernty , 

f Derelopment of Modem Society 
a An InlroduetioQ to Social Science 

E Honda State Unirernty 

A Huloncal Development of Modem 
Society 

1 An Introduction to Contemporary 
Cmluation 

F Florida Univereityof 

1 American Inetitutiont 

G Kansai SUte College 

i Intreductov Social Science 
n Kaneai State teachcri College Fittiburg 
i The Amencaa Heritage 
m CootemMrary Amenean ProbleDt 

I Mielugas state College 

« Introduction to Social Science 

J Minneeeta, Umvereity of 

c Probleou of Contemporary Society 

K Maikingum College 
• Sociu Studiea 

L. Oklahoma A &M College 

g Challengei m Ameneaii Democzttte 
life 

M Stepbeni College 

r Contemporary Social laanee 

N Syracuae Univeriify 
f Democratic Citueuihip 
t Problemi of Amencan Democracy 

0 TVngbt Junior Colley. Chicago 
«, Social Science 

■V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

V 

10 

7 

0 

0 

s 

8 

8 

B 

4 

6 

8 

5 

6 

SI 

8 

8 

e 

e 

6 

6 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

No 

Yes 

No 

? 

? 

No 

No 

No 

Yes 

No 

Yes 

No 

Yes 

1 

t 

1 Si 2 

1 

4 

e 

1 

2 

1 

1 

2 

f 

t 

1 

1 or 

1 

1 

1 

1 

2 S 

1 

Total 

9 

1 

9 

7 

8 





form of a test -were screened and all terms which seemed even 
remotely technical were used as the basis for a separate vocabulary 
test. The Test of Critical Thinking in Social Science and the Vo- 
cabulary Test were administered to the same students to test the 
hypothesis that knowledge of terminology was essential for critical 
thinking but not sufficient of itself to ensure success in critical 
thinking The several studies, to be reported later, thoroughly 
supported the hypothesis. Students with low scores on the Vocab- 
ulary Test were unable to do anything with the critical thinking 
test. Equally significant, however, was the fact that some students 
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with good vocabulary scores did very poorly on the critical think- 
ing test. 

Two other issues had to be feced at this time. The first of these 
involved the limitations of existing testing techniques. Many indi- 
viduals contend that ability to do critical thinking cannot be as- 
sessed by means of objective-type test questions. They feel that the 
presentation of several responses “gives atvay much of the show.” 
Their solution to a measurement problem would be to ask students 
to respond orally or in ivriting to a problem situation. This solu- 
tion, however, has two serious limitations. Scoring of students’ 
written or oral responses poses such a formidable task as almost to 
preclude dealing with large numbers of students. The second 
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Evaluation of the ability “to sense or to define a problem” requires 
thorough understanding of what people do in demonstrating this 
trait. The practicing educator cahnot trace the actual psychological 
or physiological process. Indeed, it is doubtful if experienced 
clinicians can do this. The teacher can only infer the existence or 
absence of the trait from the activities in tvhich students engage. 
Thus the task of developing evaluation techniques becomes one of 
cataloguing various kinds of overt student acts from which can be 
inferred demonstration of whatever traits are immediately relevant. 

Criteria for Selection of Problem Materials 

Clearly the social sciences are rich in materials which can serve 
as provocative stimuli for problem-solving activity. The available 
data with respect to communism, socialized medicine, euthanasia, 
pro- or anti-vivisection, or tidelands oil, typify the wealth of such 
information. However, not all materials impinging on these topics 
are equally adaptable for testing purposes so that in selecting ma- 
terials several criteria must be applied: 

1. A problem situation should be meaningful in terms of the 
student’s otvn experiences. It would be possible to construct a test 
of critical thinking making use of highly abstract or even nonsense 
symbols. For some purposes this is appropriate, but for a test in 
the social sciences, material more relevant to the everyday world 
sfiouM 6e sought. Turthennore, test items must have cansfeferaWe 
intrinsic interest if students are to put forth their best efforts. This 
same principle of relevancy also rules out some highly specialized 
areas of the social sciences which tvould be of interest to the spe- 
cialist but of little concern to a freshman or sophomore college 
student. 

2. Materials selected should be short enough for students to 
study in a relatively brief period of time. A reading passage, for 
example, should contain a great deal of information in a paragraph 
or so. Tabular or graphic information should be brief yet complex 
enough to present a significant task, 

3. Problem situations should involve a certain degree of emo- 
tion for most persons working with the material. Human beings 
can be objective and critical about problems which are purely 
academic, whereas this remoteness is considerably more difficult to 
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limitation lies in the unreliability of appraising written and oral 
responses This unreliability is heightened further as test questions 
attempt to go beyond the mere assessment of factual knowledge 
and enter into the domain of the more abstract intellectual skills 
The committee finally decided to prepare an objective test, in full 
a^sareness that certain elements important in critical thinking 
were not adaptable to objective testing techniques * It is possible 
to present an examinee with a paragraph and ask him to detect 
the central issue of the argument by means of an objective type 
question, but the fecundity with which an individual can invent 
hypotheses to fit a particular problem is outside the scope of an 
objective type test To test this latter trait might require the ex 
aminee to list many possible hypotheses which could explain a 
given situation ® 

The second issue involved a concern as to whether classroom 
teachers for the most part inexperienced in the theory of test 
construction, could produce respectable evaluation instruments 
dealing with relatively complex intellectual skills It was felt that, 
with some help from experts classroom teachers possessing insight 
into their subject and into the student mind, could develop ap 
propriate evaluation and research methods One of the correlative 
values of the Cooperative Study has been the in service training in 
testing techniques for a goodly number of classroom teachers 
Since the development of evaluation materials for the assessment 
of behavioral outcomes is relatively new m higher education, a 
description of how the Social Science Committee proceeded may 
be helpful in the understanding of results reported later As has 
already been indicated, the objective of increased critical thinking 
was defined by specifying some of its behavioral attributes This 
specification is the first step in the evaluation process, but scanning 
the critical abilities listed above will quickly reveal that they are 
still far removed from practical evaluation and teaching situations 


with essay testing Toward 

the end o the Study the members returned to a consideration of essav testine and 

Vwa 

P«xJ«ced several forms of a test involving 
Ind iL hi ^ Productive Thinking Guilford 

and hu aisoaaics have also done considerable research along similar linw 
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7. Does the item contain irrelevant details which might reveal the 
correct response to the students? 

8. Does the item require greater knowledge than most of the students 
could be expected to possess? 

9. Does the stem of the item present a clear problem to the students? 

10. Is the stem of the item free from ambiguities? 

11. Does the stem present its problem as briefly and as clearly as 
possible? 

12. If the stem employs the negative form, are the negative words or 
phrases emphasized? 

13. Do the foils make sense when compared with the stem of the item? 

14. Are the responses to the items freedom ambiguities? 

15. Are the responses stated as briefly as possible? 

16. Do the foils sound realistic and plausible? 

17. Are there more than the intended number of correct responses? 

18. Is the item free from grammatical and punctuational errors? 

19. Is the item free from typographical errors? 

20. Do you agree ^vith the keyed response? 

21. Does this item seem worth keeping for further work? 

Development of the Test of Critical Thinking 
IN THE Social Sciences 

The preceding discussion has su^ested two possible dimensions 
for test items. One of these dimensions might be called a skills 
dimension and the other a subject or content dimension. In plan- 
ning and assembling an examination, it is useful to have a two- 
dimensioasl grid as a kind of blueprint for the test builder Co 
follow. One axis of the grid lists the specific skills with which the 
test is concerned, in this case the list of abilities involved in critical 
thinking. The other axis lists content areas within which the skills 
are to be operative. Naturally, the content axis of the Social Sci- 
ence Committee was derived from their survey of course content. 

"With such a grid the test builder can decide a priori how he 
would like to balance his final examination. The grid brings to the 
conscious level decisions which are commonly made without con- 
scious attention and therefore without any rationale. The existence 
of the grid does not require that an equal number of items be 
assigned to each cell. The test builder may decide that he would 
like to have one item for each skill for each content classification, 
or he could decide to sketv the distribution in ^vhatever direction 
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attain i£ the problem is one which involves the individual emotion- 
ally. It would be impossible to find test materials which would 
involve emotionally all students but it is possible to pick out situ- 
ations which will prove stimulating to the majority. 

4. Materials should be intelligible to students. 

Experienced teachers often know what problems appeal to stu- 
dents, what they find challenging, and what they can solve. Teach- 
ers with less insight can obtain equally good results by asking stu- 
dents for their opinions about materials and by trying out ma- 
terials in a variety of ways on students. 

These two techniques were utilized by the Social Science Com- 
mittee in obtaining problem situations. The members solicited 
their colleagues for appropriate materials, worked these materials 
into trial testing situations, and administered them to their own 
classes. Students were asked to solve the problems and to indicate 
their opinions about the activities in which they had been required 
to engage. Their answers and opinions provided one basis for re- 
vision of materials. 


Criteria for Development of Test Items 
The creation of plausible alternative responses to an objective- 
type question is always a difficult task. Perusal of student free- 
response statements about the materials under consideration fre- 
quently proves to be a source of good ideas for possible alterna- 
tives. After test items have been developed, they may be evaluated 
by the test writer or by other competent persons using questions 
such as the following: 


1. Does the item appear to you to measure what it purports to 
measure? 

2. Does the item seem to you to be realistic and practical? 

3. Is the language of the item appropriate to the educational level 
at which it is expected to be used? 

4. Is the item or group of items capable oE standing independent o£ 
all other items of the test in which it is to be used? 

6. Docs the item contain unnecessary details or purposeless elements 
of conrntioTiS * ^ 


. Does the item appear to you to measure possession ( 
pnnaple, concept, skill, or ability? 
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he desires. The Test o£ Critical Thinking in Social Science was 
eventually prepared by the Social Science Committee on the grid 
presented in Figure 1. In this case the committee aimed at ap- 
proximately equal numbers of items for each ability, but planned 
to have about four times as many items dealing respectively with 
political and economic affairs as with the culture concept and 
social affairs. Some modification of these intentions was enforced 
later by the difficulties in building good test items. 

The process of developing good test items, particularly those 
measuring traits beyond simple recall of information, by no means 
stops when the draft items have been prepared. Items ought to be 
reviewed critically by competent colleagues before being ad- 
ministered to students. These colleagues should judge the items 
in terms of scholarship and relationship to whatever trait they 
were intended to measure. Items withstanding this critical scrutiny 
can then be consolidated into trial tests and administered to groups 
of students. At that time, students can be asked their candid 
opinions about the test materials. Student responses to the items 
can be treated statistically to determine their relative efficiency. 
Still further insight may be obtained by asking students to verbal- 
ize their thought processes for recording as they solve problems. 

All of these procedures were used in developing the Test of 
Critical Thinking in Social Science. Even so, the test went through 
several revisions before reaching its present form. This test was 
designed to sample each of the behaviors shown on the grid but 
was to yield a total score which could serve as an index of students’ 
general critical thinking ability in the social sciences. At the point 
of putting together the final test form the committee had to resolve 
an issue which constantly intrigues anyone working with tests deal- 
ing with a complex of abilities. There appears to be a strong temp- 
tation to try to obtain multiple subscores from a relatively short 
test. Teachers are prone to say, “Wouldn’t it be nice if we could 
find out in which of these skills our students are strong and in 
which weak?’’ Once having thus verbalized this hope, they are 
nudged almost irresistably toward creating subscores, often un- 
reliable and ordinarily not used. The Social Science Committee, 
after having been faced with this temptation, decided that it would 
use only a total general index of critical thinking ability. 




Fia. 1 — <3niD Fobu fob the Constbuctiom or the Test or Cbiiical TniNKiNa in SoaxL Sctence. The aim of the Social Sdence Commit- 
tee was that the number of items for each ability would be approximately equal. 
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he desires. The Test o£ Critical Thinking in Social Science was 
eventually prepared by the Social Science Committee on the grid 
presented in Figure L In this case the committee aimed at ap- 
proximately equal numbers of items for each ability, but planned 
to have about four times as many items dealing respectively with 
political and economic affairs as with the culture concept and 
social affairs. Some modification of these intentions was enforced 
later by the difficulties in building good test items. 

The process of developing good test items, particularly those 
measuring traits beyond simple recall of information, by no means 
stops when the draft items have been prepared. Items ought to be 
reviewed critically by competent colleagues before being ad- 
ministered to students. These colleagues should judge the items 
in terms of scholarship and relationship to whatever trait they 
were intended to measure. Items withstanding this critical scrutiny 
can then be consolidated into trial tests and administered to groups 
of students. At that time, students can be asked their candid 
opinions about the test materials. Student responses to the items 
can be treated statistically to determine their relative efficiency. 
Still further insight may be obtained by asking students to verbal- 
ize their thought processes for recording as they solve problems. 

All of these procedures were used in developing the Test of 
Critical Thinking in Social Science. Even so, the test went through 
several revisions before reaching its present form. This test was 
designed to sample each of the behaviors shown on the grid but 
was to yield a total score which could serve as an index of students’ 
general critical thinking ability in the social sciences. At the point 
of putting together the final test form the committee had to resolve 
an issue which constantly intrigues anyone working with tests deal- 
ing with a complex of abilities. There appears to be a strong temp- 
tation to try to obtain multiple sufascores from a relatively short 
test. Teachers are prone to say, “Wouldn’t it be nice if we could 
find out in which of these skills our students are strong and in 
which weak?’’ Once having thus verbalized this hope, they are 
nudged almost irresistably toward creating subscores, often un- 
reliable and ordinarily not used. The Social Science Committee, 
after having been faced with this temptation, decided that it would 
use only a total general index of critical thinking ability. 
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Typical Test Items of Critical Thinking 
IN Social Science 

The following items are presented as illustrative and contain 
some explanatory comments as to the skills they purport to meas 
ure They are similar to the ones included in the Test of Critical 
Thinking in Social Science Consider, for example, the following 
items 


The average American consumer, if he thinks in terms of economic 
self interest, will give primary consideration in judging the desira 
biiity of the sales tax as a means of raising federal revenue to the 
fact that ° 

1 many people own no property to tax 

2 '•'“me and corporate taxes account for a major part of the 
tederal revenue 

4 taxes have declined as a source of federal revenue. 

^ to pay ^ generally falls most heavily on those least able 

er^.tt,f” "T'l" recall some gen- 

coneeTof ^ Prof«sor He must analyze the 

va^v of L'ahr®' Then he must constder the rele- 

this hvDothet responses to the economic self interest of 

on’: Ks'rm'’"^'’" “'y prop- 

Dealing with a different skill ,s the item 
non, "Ma ifi! fsoci™ nfmal 7 '“PPo" for the generaliza- 

2 Pr'LuL't Srottp customs 

3 s aTtrnrrchZr alir ■" 

conduct is molded bv mn '-“nt, and an adult, man’s 
individual ^ influences whtch are outs.de the 

4 Human nature ts mosUy learned behavior 
Structurally man resembles odter animals.' 

This IS an Item designed to test th^. - jLiL, ^ j 

given facts tend to suooort a a™ i determine which of 
reading the stem, may fomulate^inlhe*'^'^*™ students, m 

and will then look among the responses°tIr 
different order from the fnes the^rghrof irh^ 
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quires even the poorer students to reflect. They will generally 
attempt to discover the correct answer by a process of elimination 
—which is not a bad intellectual technique. 

Although requiring considerably more factual information than 
the previous item, the next one still goes far beyond mere factual 
knowledge. 

The due process clause of the Fourteenth Amendment 

1. adds nothing to the due process clause of the Fifth Amend- 
ment. 

.2. strengthens the Fifth Amendment by exactly duplicating its 
terms and meaning. 

3. adds to the due process guarantee by placing a federally 
enforced prohibition on state action in violation of due 
process. 

4. weakens the due process guarantee by limiting the prohibition 
on state action in violation of due process. 

A student might answer this correctly by knowing that the Four- 
teenth Amendment acts on states. On the other hand he might, and 
this is more likely, solve the problem by making an analysis some- 
what in these terms; 

It isn't likely that the Fourteenth Amendment adds nothing to the 
Constitution, else it never would have been ratified. By virtue of the 
same point, response 2 is wrong because a law is a law and doesn't 
gain any force by mere repetition. Response 4 implies a weakening of 
the Federal Constitution. If I remember my history, the Fourteenth 
Amendment was written at a time when state rights were on the wane. 
This leaves only response 3 which, now that I think of it, is a point of 
fact. 

Frequently test writers find it desirable to present students with 
certain written, tabular, or graphic material to provide a basis for 
testing reading and thinking abilities. The passage below is 
followed by five questions designed to do this. 

Read the following paragraph. Then answer the questions which 
follow. 

If "peace” is conceived in its broad meaning, as the "tranquillity of 
order,” it is clear that if any individual nation or group of nations 
is to attain that happy condition, its primary task is the repression 
of violence. Here the analogy with the domestic law of the state is 
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not only suggestive but compelling. Long ago it was recognized 
that there can be no peace within the state if each citizen is to be 
allowed to take the law into his own hands and enforce his claims 
by his own armed might. Violence is forbidden him, and any resort 
to it is unlawful and punishable even though it should subsequently 
appear that the claim itself, by the judgment of the courts, was a 
just one. To the extent that the law of the state defends existing 
personal and property rights, it protects the status quo; and it does 
so even when, on occasion, there is reason to believe that abstract 
justice might call for a different decision. As between citizens, so 
between nations, the repression of violence must be the paramount 
obligation of the international community, to which all others are 
subordinate. Under no circumstances must a nation be permitted 
to take the law into its own bands; and if it should do so, it must 
find ranged against it the organized community of nations which 
will see in its act of violence an attack upon the principle of law and 
order and, therefore, an attack upon each member of the com- 
munity individually. 


I. The author of the statement above holds that the fundamental 
requisite for the establishment of the "tranquillity of order" to 
which he refers is the 

1. establishment of just laws by the iniemalional com- 
munity. 

2. elimination of aggression by single states or groups of 
states. 

3. reduction of national states to mere administrative dis- 
tricts. 

4. elimination of violence within national states. 


2. The sentence beginning with which of these words contains 
the key idea of the paragraph? 

1. I4cre the analogy with the domestic law . . 

2. ‘Long ago it was recc^ired that . . 

3. “To the extent that the law of the state . . 

4. As between citizens, so between nations . . 


3. A tasic asjumption underlying the author’s contetrtions is 

1. national self-interest must be the chief motivating factor 
in international relations. 

2. national armaments inevitably lead to war 

4 ° P^aanywhere is a threat to it everywhere. 

■ abstmc“e“ 
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4. Which, of these concepts would be least in harmony with the 
author’s viewpoint? 

1. Arbitration. 

2. Diplomatic immunity. 

3. Neutrality, 

4. Treaties. 

5. The general principle which is advocated by the author is 
known as 

1. collective security. 

2. balance of power. 

3. imperialism. 

4. multilateral agreement. 

The first item requires students to see that the author equates 
"tranquillity of order*’ with "peace." Then he must scrutinize the 
responses in terms of what is fundamental to peace in the eyes of 
the author. In the passage the author talks about just laws, but in 
t^vo places the remark is made that violence must be eliminated. 
This points to response 2 as the correct answer. 

The second item demands the student be able to sense the 
crucial element of the paragraph, but in addition to recognizing 
this central problem the student should be able to detect the as- 
sumptions upon which the central idea rests. The third item re- 
quires the student to do this. 

Then, as a further check on student understanding, he is asked 
to equate the point of view of the author with an appropriate tech- 
nique of international relations. He must know the meaning of 
these techniques to see that neutrality is least in harmony with the 
notions being considered. The last item requires the student to be 
able to^so paraphrase the general principle enunciated by the 
author as to be able to apply an appropriate generic term to it. 

Items requiring demonstration of critical thought may be based 
on various kinds of quantitative data. Data are supplied and stu- 
dents are asked to interpret given statements in the light of the 
data or they may be asked to bring previously acquired informa- 
tion to tile interpretation of the material. The table and items 
given below require interpretation without recourse to outside 
information. This entire series is designed to test students’ ability 
to recognize the adequacy of data to support certain generaliza- 
tions. 
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Examine the table below whidi shows percentage distribution of 
marriage choices in Amsterdam, Holland, during two selected 
periods. 


FeBIOD I 

1 Backeloss wm | 

WiDOWEBawmi 

Spinsters | 

Wi^ws 

Divorced 

Women 

Spinsters | 

Widows 

Divorced 

Women 

1905-1909 

1926-1930 

96.1 

95.2 

2.4 

1.5 

1 5 

3 5 

62.0 

59.9 

81.4 

2C.8 

6.6 

18.8 


P“” 1 

SrtKBTEBS WITH j 

1 Widows WITH 

Bscbelors 

Widowers 

Divorced 

Men 

Bubelors 

Widowers 

Divorced 

Men 

1905-1909 ! 

1926-1930 

92 0 
90.2 

5.9 

5 2 

2.1 

4.6 

41.6 

29.7 

53.0 

65.7 

4 5 
14.6 


Som A ‘nf'>™ation contained in the table, select 

statement about each of the following 

1. The statement is jusitfied by the data in the table. 

2. The statement is contrary to the data in the table. 

. There IS not enough information given in the table to 
either justify or contradict the statement. 

’■ “erM^Swgsr™''’ ™ >S»5-I909 than in the 

Itto ma“it fo" - 

3. About thme^ftlTs tTh" “ 

who had never been marteTS^rr''’""''" 
widoweT^^ widows was greater than the number of 

' marrM^t^wMCem! widows were 

’■ So1„™e' mndent'r^M -i ^926- 

vorced perLns ^ widowers to marry di- 

8. The divorce rate was higher in the second period than in the 



OBJECTIVES IN SOCIAL SCIENCE 55 

Summary of Research Data Concerning the Test 
OP Critical Thinking in Social Science 

Reliability and Validity 

In preparing the final form of the Test of Critical Thinking in 
Social Science, studies of item analysis, reliability, and the validity 
of the test instrument were carried out. The detailed results of 
this work are described in an instructor’s manual prepared to ac- 
company the test.* Some summary of those findings is necessary, 
however, to aid in the proper interpretation of research findings 
reported. 

The test has proved adequately reliable for purpose of group 
measurement of college freshmen and sophomores. Depending on 
the size of the group tested and the technique used to estimate the 
reliability, coefficients centered about the .80 magnitude. In view 
of the length of the test and the nature of the trait it purports to 
measure, such statistics were considered satisfactory. Item analyses, 
by which the difficulty and the discriminative characteristics of test 
items are estimated, yielded highly satisfactory results. 

The validity of the test rests primarily on the judgments of the 
committee, but a number of other techniques were used to confirm 
this judgment. These included comparisons of the test scores with 
(1) those of other appropriate instruments; (2) course grades; 
(3) outside estimates of student possession of the relevant skills; 
and (4) student demonstration of critical thinking by other means. 
The unequivocal conclusion of all of these when combined with 
expert estimate of the test’s potentialities is that the test measures a 
complex of traits which can be legitimately described as critical 
thinking in the social sciences. 

Student Growth and Development in Critical Thinking 
in Social Science 

In the early fall of 1951 the Social Science Committee completed 
a satisfactory form of its test and proposed spending the next year 
(1951-52) finding out the capabilities and limitations of the in- 
stniment. The length of the Study, however, made it desirable to 
engage in some research beyond that involved in test analysis. To 
•Copies niay be purchajed from the EduaUona] Tejtlng Senlce, rrlnccton, N.J. 
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combine these t^vo purposes, a simple research program was pre- 
pared to be carried on during the academic year 1951-52. The 
purpose was stated as a study oE growth and achievement in social 
science, as measured by the Test of Critical Thinking in Social 
Science, in relation to performance on the T est of Critical Think- 
ing, Form A, and the several tests of beliefs and attitudes. 

One major concern has been to determine something of the 
nature of the critical thinking skill. Some evidence on this is 
revealed in Tables 3 and 4. Critical thinking is positively related. 


TABLE 8 

COEmcIENTB 07 CoBIlELATlON OP THE TesT OP CRITICAL TniNKINa 

IN Social Sciencb with Other Tests 



• Quma UeNesur. SUUutut (Ne* York John WJey L Sow, IMB). pp. US-M 


although not perfectly so, with intelligence. The degree of this 
relationship is emphasized by considering the lower order of 
magnitude of tire correlations involving the Inventory of Beliefs 
and the Problems in Human Relations tests, ivhich are measures 
of more affective traits. 
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on a vocabulary test containing technical or quasi-technical terms 
found in the critical thinking test. Due to the simplicity of the 
vocabulary items, most students scored high on them, regardless of 
their score on the critical thinking test. However, there were 
sufficient lo^v scorers to present an interesting pattern. Some high 
scorers on the vocabulary test scored high on the critical thinking 
test and some scored low. Some students scored low on both tests. 
No students, however, scored low on the vocabulary test and high 
on the critical thinking test. These data, derived from three differ- 
ent studies, support the generalization that knowledge of the con- 
tent of the social science area is not of itself sufficient to guarantee 
success in critical thinking. 

Scores of the Test of Critical Thinking in Social Science are 
positively related to grades in general education courses in social 
science at about the same magnitude as are tests of knowledge or 
intelligence. Coefficients range from .53 to .68 and are in general 
slightly higher than similar statistics for academic aptitude tests 
and course grades. Perhaps as a by-product, the test may show a 
way toward better prediction in this area. One comparison t\^as 
made between critical thinking scores and grade-point averages for 
all general education courses and for all non-general education 
courses taken by a particular group of students. The r’s of .41 for 
the former and .30 for the latter are interesting, especially in the 
fight of the fact that critical thinking is made explicit in the objec- 
tives of those general education courses but not in the specialized 
courses. 

Further relationships also have been studied to show more of the 
nature of the trait critical thinking. An r of .73 was obtained 
between a reading test especially designed for general education^ 
and the Test of Critical Thinking in Social Science, an r of .56 was 
obtained between a final social science course examination and the 
Test of Critical Thinking in Social Science, and an r of .71 was 
obtained between the General Educational Development Test 
(High School)® and the critical thinking test. At one large institu- 
tion the test ■vs’as administered to a group of new students which 

* East Lansing. Mich.: Mldiigan Sutc College Board of Examiners; copyright 
1951. 

‘Tests of General Educational Development, High School Level (Princeton, N.J.: 
Educatloiul Testing Service; cop}Tigbt 1915 by American Council on Education). 
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combine these two purposes, a simple research program was pre- 
pared to be carried on during the academic year 1951-52. The 
purpose was stated as a study of growth and achievement in social 
science, as measured by the Test of Critical Thinking in Social 
Science, in relation to performance on the Test of Critical Think- 
ing, Form A, and the several tests of beliefs and attitudes. 

One major concern has been to determine something of the 
nature of the critical thinking skill. Some evidence on this is 
revealed in Tables 3 and 4. Critical thinking is positively related, 
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critically. The reasoning runs that students from the upper socio- 
economic levels, by virtue of their broader experiences and richer 
cultural level, should do better than less fortunate students. To 
test this reasoning, biographical data were collected about each 
student examined in the fall of 1951. These data were treated by 
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analysis of variance and yielded the generalization that socio- 
economic differences were statistically significant at the .01 level 
of confidence in accounting for differences in test achievement for 
male students. No significant variance ivas isolated for female 
students. By the same technique it was discovered that there were 
some differences between students grouped according to their 
proposed major field of specialization, although these also were 
slight. 

OnC’Year Gains on Critical Thinking in Social Science 

Of much more importance in considering what affects ability to 
think critically are the data contained in Table 5 These data 
indicate how students from different colleges gain over a year’s 
time \sith respect to critical thinking ability. Students from each 
institution are classified according to their pre-test performance. 

It is at once apparent that, in general, students who score low on 
the pre-test record the most significant gains on the post-test. 
"Wliether this phenomenon only illustrates the familiar regression 
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also took a placement test sampling materials from a full year’s 
course in social science The r s from two comparable groups were 
67 and 61 

Including some of the coefficients just mentioned, as well as 
others Table 4 contains a large number of r’s reported from van 
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critically. The reasoning runs that students from the upper socio- 
economic levels, by virtue of their broader experiences and richer 
cultural level, should do better than less fortunate students. To 
test this reasoning, biographical data were collected about each 
student examined in the fall of 1951. These data were treated by 
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toward the mean which seems to pervade all testing or whether it 
has more fundamental significance, is a debatable point 
Table 6 presents mean gams for students of different ability 
levels who were divided according to their pre test scores The 
numbers to the left of the table represent quartile ranking on the 
ACE Psychological Test with First representing the lowest and 
Fourth the highest The numbers at the top of the table repre 
sent quartile ranking according to pre test score on the Test of 
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academic aptitude are held constant The cause of these vanations 
IS largely conjectural, although some of the subsequently described 
studies may provide insight on this matter 

Critical Thinking Performance of More Advanced Students 
For the most part, students whose scores were included in the 
preceding data took social science courses in their freshman year 
The resulting gams, while generally not large, were of the kind to 
be expected It then became desirable to find out what happened 
to those students during the next year of college To determine 
this, identical cases were selected who had been tested in the fall of 
1951, spring of 1952, and spring of 1953 with the Test of Critical 
Thinking m Social Science Arranged m this order the data in 
Table 7 reveal what happened 

Replication studies were made at several other institutions, the 
results of which arc included in Tables 8, 9, and 10 
From these data emerges the fact of considerable importance 
that scores on the Test of Critical Thinking in Social Science 
continue to increase, or at least remain constant, during the second 
year Forgetting of knowledge begins almost immediately after a 


TABLE 7 

Retest of Soinj Stodents Tested m Tables 5 akd 6 
OK Cbiticai. Tbikkiko IK Social Sciekce 


TetUng Dale 

a 

Mru Score 

SJ> 

Fall 1951 

81 

25 12 

7 85 

Spring 1952 

87 

81 54 

6 03 

Spring 1953 

81 

84 94 

7 24 


TABLE 8 

Perfoiuiakce or Matoied Gnocre or Students on Test or 
CnmcAL TnLVKiKa a Sooal Sctekcb 


! 

Gaocir 


EoraTXD BT ACE 

PsTCBOLOOICA!.! XAUZSa.nO'f 1 
ToraLSeoBB 

Acmrrmi'rr ih Ti»t ot 
Cbitic*!. Thi'ciiho rf 
Social *s3*j*cb 

Mew 

SJ) 

1 Meao 

S£> 

Fre-freshmen 9/51 

55 

118 

10 

26 6 

7 7 

Pre-sophomores 9/51 

55 

118 

10 

20 0 

6 5 

Posl «ophoraorcs 6/52 

53 

118 

10 

81 5 

7 2 

Seniors 5/52 I 

55 

118 

10 

85 8 

5 7 



62 


GENERAL EDUCATION 


learning experience.® Perhaps the more fundamental skills such as 
critical thinking have a more lasting effect. The problem still 
remains, however, of increasing the magnitude of the residual 
ability. 

In interpreting these figures it may be well to inquire as to the 
TABLE 9 

Test and Retest Scores on Ciutical TniNKiNa in Social 
Sctence or Sixteen Students 
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analyses have been made comparing students at the same institu> 
tion who had different but relevant course experience. The results 
from this research have suggested that minor variations in course 
experience make no appreciable difference in student achieve- 
ment. At one institution a class was divided into six groups, each of 
which carried out a different set of activities, some of which in- 
volved use of audio-visual aids and some of which did not. While 
the differences in gains for the various groups were not statistically 
significant, the experimenter felt that the use of richer course ma- 
terials did improve the quality of class work as assessed by more 
subjective means. 

At another institution pre-test and post-test mean scores were 


TABLE 11 
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compared for three groups of students: (1) those having taken a 
general education social science course, (2) those having taken an 
introductory history course, and (3) those having taken neither 
social science offering. No significant differences between groups 
were found, although the group taking no social science course 
made smaller gains than the other two groups. Table 1 1 indicates 
this. 

Still another institution provided data showing comparisons 
between tliree groups of students divided on the basis of the pat- 
tern of general education courses they took. These data revealed 
no significant differences between students who had and those 
who had not taken the general education social science courses. 

At another institution one member of the Social Science Com- 
mittee attempted to use a series of special teaching techniques iriih 
one section, ivhile only die more customary ones were used in two 
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other sections of the same course. His purpose was to determine 
whether or not these special techniques appreciably affected stu- 
dents’ achievement on the Test of Critical Thinking in Social 
Science. Unfortunately he did not teach the control sections (2 
and 3 in Table 12); hence the results are open to question. Table 
12 suggests that at least the particular technique employed did not 
make much difference in test achievement. 
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needed and have focused attention on the significance of the kind 
and quality of collegiate teaching. 

The evidence shows that there is growth, over a period of a year, 
and over a period of two years, with respect to the traits assessed by 
the Test of Critical Thinking in Social Science and that different 
groups grow at different rates. This suggests that it is possible to 
plan learning situations so as to facilitate development of critical 
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thinking ability. How this might be accomplished is still largely 
conjectural, although some clues have been provided. 

Development of an Instructional Handbook 
As the Social Science Committee worked through the steps in 
the evaluation process, its members became aware of some of the 
limitations of existing teaching practice. The problems developed 
for tests were good ones, requiring a high degree of intellectual 
skill; yet when the problems were compared with what students 
were required to do in classes, they proved that little was being 
done to teach for critical thinking Such an awareness caused the 
members to view the project m which they were engaged in a new 
light. And they began to seek ways by which their activities might 
be more intimately related to the classroom. As a visible manifesta- 
tion of this concern a handbook ivas prepared describing various 
ways by which critical thinking abilities might be taught for in 
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social science courses. This book presents a point of siew which 
might be worth summarizing.*® 

The basic assumption is that critical thinking may be developed 
more fully by requiring students to practice the skill frequently 
mthin the context of social science content. If the desired skills 
are represented by the tasks posed by the test questions, then such 
problems can as well he utilized in teaching as in evaluating. Such 
adaptauon. however, should not take the form of mere drill on 
the solving of test questions, but should seek to develop in students 
a hicility for handling a variety of problems. The process of critical 
thinking, not the answers to specific questions, should be the goal 
of instruction. ° 
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factors associated with them. In general, it was found that students 
gained in ability to think critically in social science over a period 
of a year, although the size of these gains varied widely, depending 
on the institution that students attended. Attempts to teach 
critical thinking in social science by making minor changes in 
particular courses did not appear to result in greater growth than 
was found in courses not making overt attempts to teach this skill. 
Attempts to relate growth in critical thinking ability to course 
organization or to specific teachers suggested that both of these 
were highly Important, although the research could not identify 
specific factors tvhich seemed to be operative. 

Feeling that more attention should be given to the teaching of 
critical thinking in the context of social science, the intercollege 
committee prepared a handbook describing the teaching and evalu- 
ation of this trait in social science courses. This handbook outlined 
a rationale of thought and presented many examples of testing or 
teaching situations which might be adapted to almost any social 
science course. The experience of the intercollege committee 
members suggests that explicit attention to critical thinking can 
result in a much richer, more varied approach to teaching, with 
the possibility of infinitely greater student growth. 
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social science courses. This book presents a point of view which 
might be worth summarizing." 

The basic assumption is that critical thinking may be developed 
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(Dubuqae, Iowa: Wm. C. Erowo Co., Iffiq. I"' ^wlool.oo ood T.cch.ng 
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social science courses This book presents a point of view which 
might be worth summarizing “ 

The basic assumption is that critical thinking may be developed 
more fully by requiring students to practice the skill frequently 
within the context of social science content If the desired skills 
are represented by the tasks posed by the test questions then such 
problems can as well be utilized in teaching as in evaluating Such 
adaptation however should not take the form of mere dnll on 
the solving of test questions but should seek to develop m students 
a facility for handling a variety of problems The process of critical 
thinking not the answers to speciBc questions should be the goal 
of instruction 


The impact such a point of view will have on college teaching 
♦u ? unknown However there is some reason to suppose 

actively participated in the Cooperative 
Study developed an increased awareness of the limitations of their 
their ‘“'ght into other ways of approaching 


Summary 

tor'^geneXrtr? ' frequently claimed 

SmZtee nn s ' ‘he Intercollege 

m social science speS h"® 

of their con™ « 

tried out on students various kinds'lV" 

objective type Test of Critical Th, f techniques an 

developed revised and Thinking in Social Science was 

ministered to a teafm™ “ * This test was ad 

participating colleges and to freshmen at a number of 

groups at the end of the freshmaT or to comparable 

more year The changes :fre:t ~ “P'’“ 

were studied with a view tnA^ ^ these time intervals 

T, 1 *‘«™*nmg Iheir magnitude and the 

(Dubuque Iowa Wm C. towl, HoodbooS /or and Teach, ng 
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attempts to provide a completely unified or integrated course 
including all skills.^ 

This common concern is reflected in statements of specific course 
objectives "which were accepted, at least in principle, by representa- 
tives of institutions participating in the Intercollege Committee on 
Communications Objectives, although not all those institutions 
possessed unified courses. All agreed that students should under- 
stand sentence structure and how it reflects relationships of ideas; 
that students ought to be able to listen with concentration and 
judgment; and that they should understand the role language 
plays in influencing social behavior. They also agreed that students 
should develop skill in logical thinking; that they should be able 
to read with critical comprehension; and should be able to engage 
in group discussions or face-to-face conversations with profit to 
themselves and others. 

The committee made a brief survey of the present state of 
evaluation activity in communications and found that; 

1, Effective testing of reading ability is important and much 
already has been done about it. However, virtually nothing has 
been done on a large scale to determine growth in reading ability 
as a result of collegiate experiences. 

2, Most testing of reading ability in the past has been restricted 
to questions about what an author explicitly said or what could be 
logically inferred from his remarks. There have been some at- 
tempts to appraise sensitivity to nuances, overtones, and values in 
literary works. In addition, there is need: to assess the ability to 
analyze fairly complex'^ arguments; to recognize assumptions, hy- 
potheses, reasons, and conclusions; to detect lack of evidence, fal- 
lacies, and loopholes; to estimate relative strength of arguments; 
and to see how arguments can be refuted. At this point tiie com- 
mittee ivas clearly concerned with what otlier committees identi- 
fied as aspects of critical thinking. 

3, Ability to ivrite effectively is usually appraised through evalu- 
ation of rsnritten work, but objective-type devices have been used to 
measure this skill. Each technique possesses inherent weaknesses. 

‘Earl J. McGrath (ed.). Communication in Central Education (Dubuque, lou-a: 
Wm. C. Drown Co., 1W9). 
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—the more valid one? Would it be feasible to develop devices for 
evaluating effective or ineffective communication regardless of 
whether oral or written symbols were employed? 

Some work done by psychologists during and since World War 
II appeared to possess potentiality in this regard. Faced ivith the 
need for evaluating the effectiveness or ineffectiveness of individ- 
uals in various kinds of ivork situations. Dr. John C. Flanagan, of 
the American Institute of Research, developed a new technique for 
approaching such a problem known as the "Critical Incident 
Technique.”* 

The basic data of the technique consist of descriptions of effec- 
tive and ineffective behavior reported by supervisors who have 
been observing the workers and making judgments about the 
effectiveness or ineffectiveness of their behavior. The reports are 
examined and the "critical” behaviors are separated from other 
descriptive materials in the incident- Next, a classification system 
is derived which will facilitate the grouping-together of similar 
behaviors. The third step is to study carefully each critical be- 
havior and classify it under one of these categories. The final step 
is to wite descriptive statements which accurately summarize the 
behaviors tvithin each category. It is often necessary to revise the 
classification system and repeat the ordering of data until the re- 
sulting statements are of a nature appropriate to the purposes of 
the study. 

It seemed to the committee that Flanagan's approach might be 
suitable for determining the critical requirements for effective 
communication. If a sufficient number of critical incidents, descrip- 
tions of effective or ineffective behavior of students in communica- 
tion situations, could be collected and classified inductively %vith- 
out direct reference to the logic of either speech or tvridng subject 
matter, some indication might be given of the degree of unity of 
the communication process. 

During the spring of 1950 teachers at participating institutions 
were asked to collect critical incidents either of writing or of 
speech. Incidents of effective or ineffective listening and reading 
^v^ere excluded because of technical difficulties; however, as a side 
project some data were obtained concerning student listening. To 

* John C. Flanagan, Critical Requirements for Research Personnel (Pimburgb, Pa4 
American Inatituie for Research, 1W9). 
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If imtmg IS judged, the chief problem lies in the subjective evalua 
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—the more valid one? Would it be feasible to develop devices for 
evaluating effective or ineffective communication regardless of 
whether oral or witten symbols were employed? 

Some work done by psychologists during and since World War 
II appeared to possess potentiality in this regard. Faced with the 
need for evaluating the effectiveness or ineffectiveness of individ- 
uals in various kinds of work situations, Dr. John C. Flanagan, of 
the American Institute of Research, developed a new technique for 
approaching such a problem known as the "Critical Incident 
Technique.’’* 

The basic data of the technique consist of descriptions of effec- 
tive and ineffective behavior reported by supervisors who have 
been observing the tvorkers and making judgments about the 
effectiveness or ineffectiveness of their behavior. The reports are 
examined and the "cricical’’ behaviors are separated from other 
descriptive materials in the incident. Next, a classification system 
is derived which will facilitate the grouping-together of similar 
behaviors. The third step is to study carefully each critical be- 
havior and classify it under one of these categories. The final step 
is to rvrite descriptive statements which accurately summarize the 
behaviors within each category. It is often necessary to revise the 
classification system and repeat the ordering of data until the re- 
sulting statements are of a nature appropriate to the purposes of 
the study. 

It seemed to the committee that Flanagan’s approach might be 
suitable for determining the critical requirements for effective 
communication. If a sufficient number of critical incidents, descrip- 
tions of effective or ineffective behavior of students in communica- 
tion situations, could be collected and classified inductively with- 
out direct reference to the logic of either speech or writing subject 
matter, some indication might be given of the degree of unity of 
the communication process. 

During the spring of 1950 teachers at participating institutions 
were asked to collect critical incidents either of writing or of 
speech. Incidents of effective or ineffective listening and reading 
^v'ere excluded because of technical difficulties; however, as a side 
project some data were obtained concerning student listening. To 

* John C. FTanagan, Critical tiequirements for Research Personnel (Pittsburgh, Pa.: 
American Institute lor Research, I9t9). 
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ensure uniformity of collection, detailed instructions were mimeo- 
graphed for teachers participating in the project and special cards 
distributed upon which incidents were to be recorded Regardless 
of whether they taught in communication courses or in some other 
area, teachers were informed m writing 

’"i 'Tf “ °PI”«unity to examine the wnting of 
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sufficient detailed description to make clear the basis of judgments 
reported; (2) reporting of evaluative statements only ("Organiza- 
tion poor," “misspelled”); (3) no identification by the instructor of 
what he considered the critical behavior; (4) incidents which led 
to the inference that the instructor was reporting behavior critical 
for determining a grade in “English Class” or "Speech Class” 
rather than behavior which was critical with respect to communi- 
cation; and (5) sets of incidents which suggested that instructors 
were sacrificing quality of incident to quantity and reporting in- 
significant incidents in order to complete twenty cards. 

Although the collected incidents were not as free from error as 
had been hoped, they were analyzed and classified by the entire 
committee. To fill major gaps which seemed to exist in the cate- 
gories thus established and to verify the rubrics already determined, 
a second collection of incidents was made with greater care being 
exercised in schooling teachers as to what they should seek. The 
thus augmented file of incidents was then reclassified into two 
tentative lists— one of examples of effective and one of examples 
of ineffective communication. Items were included regardless of 
svhether they had been discovered in the context of speech or of 
writing. While some of these clearly represented their origin, 
enough referred either to speaking or to writing as to suggest the 
interdependence of the two skills. Unfortunately, it is much easier 
to find incidents of ineffectiveness than the converse; hence the 
positive side is perhaps insufficiently represented. 

EFFECTIVE AND INEFFECTIVE BEHAVIOR 
IN SPEAKING AND WRITING 

Effective Behavior Ineffective Behavior 

I. Adaptation to the Communication Context 
A. Adaptation to the Requirements of the Assignment 
Thi student The student 

Limited subject in relation to lime Chose subject too broad for time 
a\ailable. available. 

Selected novel topic related to the Failed to observe time limits, 
situation; spoke about “public Failed to develop subject sufficiently 
speaking at the turn of the cen- to use time available, 
tury" in a speech class. Copied material without giving refer- 

cnccs or using quotation marks. 
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Effecure Behavior Ineffective Behavior 

Failed to include original thinking as 
required by the assignmenL 
Included “value judgments” in a list 
of 'details objectively verifiable " 
Submitted selected quotauons where 
precis was assigned 
Developed irrelevant topic after 
clearly stating major thesis 
Faded to treat subject in accordance 
with stated focus of the assignment 


B Adaptation to Audience and Occasion 


The student 

Handled personal subject v.itb un 
usual objectivity, dignity, and di 
redness 

Distinguished between feelings and 
their occasion 

Dressed for the occasion 

Related material to everyday expen 
ence of audience 

Coordinated verbal and visual ele 
menu in speech 

Made sketch on blackboard before 
class began so that there was no 
interruption of communication 

Used vivid figures of speech (avoided 
cliches) 

Used an effective (appropriate) at 
tention-getting device 


Used manner directly m contrast to 
preceding speaker 
Related talk to preceding one 
Used visual aid presented by oppos- 
mg speaker to make ovra point. 


The student 

Used excessive negative cnucism ir 
mating to audience 
Used offensive material— vulgar story, 
sentimentality 

Chose subject s\hich embarrassed him 
self— unable to complete talk at first 
attempt 

Dressed inappropriately for occasion 
Laughed with audience when “poker 
face” was appropriate 
Apologized for performance 
Bragged 

Misspelled one word four different 
ways. 

Submitted mussy paper 
Wrote illegibly 

Affeaed emotion— paid attention to 
style rather than meaning 
Failed to allow audience to “set” be- 
fore beginning speech 
Used cliche and trite phrases. 

Fafled to integrate demonstration* 
stopped talking to make sketch, 
passed pictures and lost audience 

Immediate Situation 
The student 

Forgot speech when interrupted 
Memorized and failed to change as 
required by immediate context. 
Forgot memorued talk- 


C Adaptation to the 

The student 

Was not disturbed by interruption 
of speech 
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II. Structure 
A. Ordering of Material 

Effective Behavior Ineffective Behavior 

The student The student 

Suited organization to purpose: Arranged material in random order: 

summary first in inspection report. rambled, 

topical ordering ior essay examina- no organization apparent. 

tion. Adopted inappropriate organization: 

general to specific. chronological order required but 

interrelated sequence. not used, 

climactic, 
psychological, 
logical. 

several points of view. 

Adopted unusual method of organiza* 
tion. 

I 

B. Paragraph Unity 
The student 

Related random series of details as a 
“paragraph." 

Included irrelevant material in para- 
' graph. 

Switched from one idea to another in 
a single paragraph* 

Wrote two paragraphs where there 
should be only one. 

Failed to order sentences logically in 
paragraph. 

Included copied material without 
showing its relationship to central 
idea. 

C. Sentence Structure 

The student The student 

Used skillful subordination. Used involved sentences: 

Used variety of sentence structure. words scrambled like vegetables in 

Used effective parallelism. a soup. 

words, words, words meaning noth- 
ing. 

Used sentences of single pattern so 
that relative importance of ideas 
was not made clear. 

Used faulty parallelism. 

Used dangling participles. 


The student 

Fitted quotation into development so 
that relationship was clear. 
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ESecave Behavior IneHective Behavior 

Used pronouns and demonstrate 
adjectives ivhich lacked clear refer 
ents 

Wrote sentence fragments 

Misplaced modifiers so that meaning 
was confused 

Introduced more details than could 
be clearly related in a single sen 
tence 

Included two apparently unrelated 
ideas in a compound sentence 

Misplaced sentence parts so that 
meaning was confused 

Failed to show relationship of sen 
tence parts 

Appended idea to a sentence rather 
than integrating it into the sen 
tence 

Made errors m punctuation 


The itudent 
Used special devices to guide reader 
underlined subtopics 
numbered points 

Effected transition by repeating key 
Vrords of two paragraphs m con 
necting sentence. 


D Divisional Relationships 
The student 


Provided no transition between para 
graphs 

ideas presented like * clothespins on 
a line ’ 

Pro\ided no transition between large 
sections 

Wrote faulty linking sentence 


III Devhopment 
A Dmlopmcnl of m Introduction 
The student 


The student 

Had adequate intniducuon to tom 
munication 

Defined key words in many ways 
Ga>e essential background 
Antiapated possible misinterpreta 
tions and dealt with them 
Used personal experience to mtro. 
duce subject. 


The student 


Had adet^uate summary 
restated mam points. 


B Development of an Ending 
The student 


Made introduction too long to be 
consistent with rest of paper 
Assumed speafic knowledge which 
could not be expected of audience 
Used references without antecedents 
(«n introduction) 

Used a beginning which worked 
counter to central purpose of essay 


Had no summary where one was 
needed. 
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Effective Behavior Ineffective Behavior 

Stopped -without adequate ending: 
said, "That’s about all." 


C. Selection of Supporting Thought Groups 
The student The student 

Covered subject completely: Failed to include essential subtopics, 

included all essential subtopics. 


D. Integration of General and Particular 


The student 

Supported generalization hy specific 
example: 

used quotation to support gener- 
alization. 

used "real examples." 

Used specific details to build up or- 
ganization. 

Selected pertinent evidence to sup- 
port judgment. 


The student 

Stated generalization without sup- 
porting facts, illustrations, or ex- 
amples. 

Gave details without pointing to their 
meaning. 

Failed to give reasons to substantiate 
judgment. 

Failed to indicate central idea to 
which illustrations were related. 

Assumed a cause and effect relation- 
ship but presented no evidence for 
the existence of such a relationship. 


£. Harmony and Helevancy of Details 


The student 

Used pertinent details. 

Used carefully selected, realistic de- 
tails: 

used statistics, 
used apt quotation. 

Identified key factors and related 
others to them. 

Used details to focus on one aspect of 
the subject. 

Used unusual distinguishing details. 


The student 

Used insufficient pertinent details. 
Used excessive details (boring). 
Failed to select and emphasize: 
all details given equal weight 
faulty relation of material to cen- 
trzj purpose. 

Failed to distinguish main ideas from 
subordinate thought. 


F. Reasoning and Originality 


The student 

Selected from his experience perti- 
nent materials to make his own 
points. 

Presented fresh or original ideas 
about the subject. 

Made clear limitations of his aigu- 
ments and qualified his opinion. 

Noted significance of lack of data to 
support a particular point of view. 


The student 

Failed to distinguish between facts 
and opinions. 

Used faulty reasoning: 
argued by applying labels, 
a non-sequitur. 

Reasoned from the specific to the gen- 
eral without proving the validity 
of such reasoning. 

Made illogical inference leading to 
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Effective Behavior 

Made a key distinction between symp* 
tom and cause 

Asked pertinent question which fo- 
cused the discussion 


Ineffective Behavior 
faulty coordination of causal classes 
Developed argument on the basis of 
an unqualified assumption 
Failed to show relationship of new 
idea to material already presented 
Stated verbally a logical connection 
which did not exist in fact. 


G. Use of MapSf Charts, and Visual Aids 
The student The student 


Used model to facilitate technical ex- 
planation 
Used visual aids 
Used pertinent demonstration 
Made demonstration with real props 


Failed to provide needed demonstra 
tion 

Failed to show relationship of demon 
stration to topic. 


H Adequacy and Recency of Sources 
The student 

Suted views at variance with facts 
readily available 
Used invalid source material: 
out of date 

questionable authority. 

IV Skill m Expression 


A Word Choice 


The student 

Used specific words to convey sense 
impression 

Used exact words. 

Seleoed words which supported one 
another and conveyed a single im 
pression 

Used proper connectives to indicate 
transition 

Used clear, racy diction 

Used "double-edged' vocabulary to 
express irony 

Adapted diction to subjcrt matter 


The student 

Used words with wide general mean 
iiig to convey narrow personal 
meaning 

Used vague general words 
used ' something’ in place of exact 
word 

Used incorrect connective 

nevertheless ’ for ' in addition to 

Misused word 

undertakings" for “activities ' 
turbine’ for ‘ turban ’’ 

Misspelled word so that meaning was 
distorted 

Used word to express meaning word 
does not usually express 
^ story * for “essay 
^^”5 forth’ for ‘express “ etc 

UsM many words where a few would 
do the job better (wordiness) 
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Effective Behavior 

Made a key distinction between symp- 
tom and cause 

Asked pertinent question which fo- 
cused the discussion 


Ineffective Behavior 
faulty coordination of causal classes 
Developed argument on the basis of 
an unqualified assumption 
Failed to show relationship of new 
idea to material already presented 
Stated verbally a logical connection 
which did not exist in fact. 


G. Use of Maps, Charts, and Vtsual Aids 

The student 


Used model to facilitate technical ex 
planation 
Used visual aids 
Used pertinent demonstration 
Made demonstration with real props 


The student 

Failed to provide needed demonstra 
tion 

Failed to show relationship of demon 
stration to topic. 


H Adequacy and Recency of Sources 
The student 

Stated views at variance with facts 
readily available 
Used invalid source material, 
out of date 

questionable authority. 


IV Skill in Expression 

A Word Choice 


The student 

Used specific words to convey sense 
impression 

Used exact words 

Selccled word, which lupponed one 
another and conveyed a single im 
pression 

Used proper connectives to indicate 
transition 

Used clear, racy diction 

Used ’‘double-edged* vocabularv to 
express irony ^ 

Adapted diction to subject matter 


The student 

Used words with wide general mean 
ing to convey narrow personal 
meaning 

Used vague general words 
used "something ’ in place of exact 
word 

Used incorrect connective 
"nevertheless ’ for "in addition to ' 

Misused word 

' undertakings" for "activities ’’ 

' turbine" for "turban " 

Misspelled word so that meaning was 
distorted 

Used word to express meaning word 
does not usually express 
“story ' for "essay " 

* bring forth” for ‘express ” etc 

Used many words where a few would 
do the job better (wordiness) 
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E0ective Behavior InefFecUve Behavior 

Misused words so that the reader was 
amused in a serious situation. 
Repeated words and phrases unneces- 
sarily. 

Misspelled many common words: 
conveyed impression of illiteracy 
through misspellings. 

Used pronunciation inappropriate to 
situation: 

“candidate/’ “dose.” 

Failed to understand connotative 
meanings of common words. 

S. Rhythmic Devices 

The student 

Used onomatopoetic words. 

C. Voice Quality 

The student The student 

Used soft, well-modulated voice. Used mechanical speech. 

Used voice to support sense value of Spoke too loudly, 
words. Spoke inaudibly. 

Mumbled. 

Spoke too rapidly. 

Spoke with excessive hesitation. 

Spoke with excessive dramatic pauses. 

D. Gesiuves ofxd Bcdy Pcsittre 
The student The Student 

Maintained active body. Used distracting bodily movements. 

Used natural gestures. Used artiHcial gestures. 

Had “sparkling eyes and lighted Had no bodily activity, 
face.” X.acked eye contact. 

After the categories were established, the next step was to write 
descriptions of each which would encompass all of the incidents 
under each heading and at the same time provide a generalized 
base for classifying additional incidents and making judgments 
about the quality of products. Following this, the task became that 
of developing some form or forms of rating scales which might be 
tested by application to new speech or writing situations (see 
Figure 2). In order to explore whether or not speech needs were 
essentially different from writing needs, two forms of the check list 




number vhlcb roprosonts yotir rating in each case 






Fic. 3.— Scale for Evaluating Speaking 
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by a teacher or classmate. They all involve utilization of an ap- 
propriate device to give the communication perspective for the 
reader or listener. At any rate, considerably more research than 
has thus far been attempted is in order before such possibilities 
can be rejected. 

2. The check lists are long and somewhat more difficult to use 
than the kind usually found in college courses. Compare, for 
example, the length and complexity of the two critical incident 
rating scales with the following two in use at one of the partici- 
pating schools: 

Speech Rating Scale Theme Rating Scale 

1. Fluency Conventions of grammar 

2. Physical and vocal control Sentence structure 

3. Organization Diction 

4. Content Organization 

5. Sense of communication Content 

Such check lists will appear much more convenient than will the 
longer ones to busy classroom teachers facing 150 students each 
week. Yet there is a price for such convenience in the form of un- 
reliability of rating. For example, the two short scales were used by 
ten pairs of raters, each pair judging twenty-five themes. The 
correlations between raters in each pair ranged from .30 to .90, 
with the average falling at about the ,55 level. A partial reason for 
this relative unreliability lies in the relative abstractness of the 
rating scale. Because the behaviors to be judged under each major 
rubric have not been specified, each teacher is free to interpret the 
concept as he will. This free interpretation can be rectified by 
providing each pair of raters with several discussion periods to 
coordinate their meanings before themes are actually rated. A 
longer, more specific scale attempts to accomplish the same end by 
different, more economical means. 

3. There was some feeling on the part of committee members 
that perhaps the entire critical incident approach was inappro- 
priate to the area of communication because of the invalidity of 
a basic assumption. The technique was originally developed in the 
context of activities for which existed clearly demonstrable criteria 
of success. Activities or incidents could be selected on the basis of 
whether or not they affect success. The criterion for fighter pilot 
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were prepared, and a modification developed for speech rating 
exclusively (see Figure 3) 

Each of these two scales provided a means by which every be 
havior listed might be judged qualitatively It was hoped that the 
large number of specific behaviors listed on the scales would not 
prove frustrating to users by their very number and that they 
would enable a rater to render a more reliable judgment about the 
act he observed 


The Items on either rating scale (Figures 2 and 3) will not 
appear appreciably different from those found on any similar a 
prion established device However, by virtue of their empirical 
origin, the categories provide a valid criterion with which to com 
T based more on logic and deduction These 
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by a teacher or classmate. They all involve utilization of an ap- 
propriate device to give the communication perspective for the 
reader or listener. At any rate, considerably more research than 
has thus far been attempted is in order before such possibilities 
can be rejected. 

2. The check lists are long and somewhat more difficult to use 
than the kind usually found in college courses. Compare, for 
example, the length and complexity of the two critical incident 
rating scales with the following two in use at one of the partici- 
pating schools: 

Speech Rating Scale Theme Rating Scale 

1. Fluency Conventions of grammar 

2. Physical and vocal control Sentence structure 

3. Organization Diction 

4. Content Organization 

5. Sense of communication Content 

Such check lists will appear much more convenient than will the 
longer ones to busy classroom teachers facing 150 students each 
week. Yet there is a price for such convenience in the form of un- 
reliability of rating. For example, the two short scales were used by 
ten pairs of raters, each pair judging twenty-five themes. The 
correlations between raters in each pair ranged from .30 to .90, 
with the average falling at about the .55 level. A partial reason for 
this relative unreliability lies in the relative abstractness of the 
rating scale. Because the behaviors to be judged under each major 
rubric have not been specified, each teacher is free to interpret the 
concept as he will. This free interpretation can be rectified by 
providing each pair of raters with several discussion periods to 
coordinate their meanings before themes are actually rated. A 
longer, more specific scale attempts to accomplish the same end by 
different, more economical means. 

3. There ivas some feeling on the part of committee members 
that perhaps the entire critical incident approach was inappro- 
priate to the area of communication because of the invalidity of 
a basic assumption. The technique was originally developed in the 
context of activities for which existed clearly demonstrable criteria 
of success. Activities or incidents could be selected on the basis of 
whether or not they affect success. The criterion for fighter pilot 
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success is whether he shoots dotvii enemy planes and is not shot 
down himself. The criterion of success for a surgeon is whether his 
patient lives and recovers health. Criteria of success for processes 
of communication are not so clear cut. If formal excellence of a 
piece of written work is the criterion, then certain incidents such 
as grammatical use are relevant. If, however, the criterion is 
rejected, as it is by many theorists about communication, knowl- 
edge of grammar loses some of its potency. This position is ex- 
pressed in an evaluative statement submitted by one member of 
the committee; 
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direct attention to other criteria accepted in practice if not in 
theory. 

4. One reason why the critical incident materials were not em- 
ployed more extensively falls under the heading of public rela- 
tions. Teachers were asked to collect the critical incidents by means 
of a mimeographed letter and instructions. This device did not 
properly convey the nuances and implications of the study without 
which the collection of data would seem to be just busywork. Then 
when the final list of categories and the final check lists appeared 
with some gaps suggested by logic, teachers became convinced that 
further work on the project would be a waste of time. 

In the light of these reasons the Communications Committee 
reached the conclusion that the critical incident attempt to study 
communication was not successful. The attempt did, however, 
have important by-products in the form of other committee 
activities. 


The Theme Analysis Handbook 

In order to practice using the critical incident check lists, the 
Communications Committee collected representative study themes 
from each of the participating schools. The idea was that these 
could be reproduced and each member of the committee could 
rate each theme by means of the rating scale. As members began 
this work, however, it became apparent that individuals ap- 
proached theme rating from many diverse points of view regarding 
the process of theme evaluation. It further became clear that these 
dilferences regarding the nature of the rater’s role made significant 
differences in the value assigned a particular paper. To reconcile 
these divergent orientations, the committee members began to 
study the process of theme rating. 

From this discussion emerged a set of principles or a rationale 
governing evaluation of student written work, the effectiveness of 
which could be demonstrated on the themes collected. These prin- 
ciples attained increased significance in the eyes of the committee 
members when they realized that in spite of handbooks of usage 
and training in ^vriting, few teachers approached theme reading 
from a consistent frame of reference. Thus, a project designed to 
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provide assistance to the committee members alone came to be 
viewed as having importance for all communication teachers, par- 
ticularly new staff members. These principles have been embodied 
in another publication* which presents them and then demon- 
strates their applicability to a collection of themes, each of'iwhich 
was critically analyzed and judged by the entire committee. Since 
they do represent the mature judgment of the committee with 
respect to one of its major concerns, they are also included in this 
report. 


Some Principles of Theme Analysis 
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tolerant. Nor does the fact that the teacher can realize the inten- 
tion of a theme automatically earn it credit. That intention may be 
worthless. Once recognized, however, the special aims that govern 
the writing should be considered as part of the total performance 
to be evaluated. 

3. Unrealized potentialities.—The reader of a theme should be 
aware of its unrealized potentialities, its opportunities wasted or 
used without imagination. His evaluation should consider not only 
what has been done but also what might have been done with this 
or that detail, image, and argument. Not only does this obligation 
on the part of the reader add a frame of reference complementary 
to the preceding two; it also gives a valuable stimulus to the stu- 
dent as he reads over the criticism of his work. 

Suggestions two and three, both touching on constructive criti- 
cism, need to be modified by the warning that their successful 
adoption requires considerable tact and caution. If the instructor 
follows the second suggestion without this caution and attempts 
to reconstruct the internal motivation of a theme, it is quite pos- 
sible that he will credit the student with and impose upon him 
ideas which the theme never contained. Likewise, an engrossing 
interest in the unrealized possibilities of the writing may make the 
teacher lose sight of what has been achieved; may make him sub- 
stitute for the student’s theme an imaginary essay with an alien 
point of view. Although it is difficult to know how consciously an 
author is aware of the intention of what he is writing, nevertheless 
the theme reader should avoid the risk of distorting another’s 
work; he should stick to concrete evidence. 

4. Interdependence of parts.— The teacher should remember 
that the paper he is reading is not a set of miscellaneous details but 
a single theme whose parts are interdependent. Their significance 
arises from their belonging to a particular order of verbal events 
and from their relation to one another within this order. To see 
them accurately is to see them in action; issuing from and leading 
to; doing things. They cannot properly be taken from their whole 
and assessed in isolation. 

However obvious this may seem, it cannot be too strongly urged. 
Despite lip service paid the proposition that every element of a 
theme is one term in a relation or in a number of relations, the 
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temptation in critical practice is to deal with these elements 
separately as they are encountered in the reading The deader 
usually does not hold in his mind the entirety of the them^ he is 
appraising because he has time to read it only once Nevertheless, 
the attempt to reread should be made lest the final evaluation be 
made on only a fragment of the original theme i 


The structural importance of the large elements of a theme— of 
introductions and conclusions, of paragraphs and key sentences— 
IS clear They can hardly be seen except in connection with the 
developments to which they contribute Yet matters of diction 
should also be examined in terms of the context in which they 
occur Logic grammar, and semantics allow the observation of 
relationships between things named the union of word to word, 
and the relation of word to thing or to the concept of a thing The 
pod theme reader will also be concerned with questions of the 
harmony of phrasmg-the way words Ht with each other and with 
the total tone and purpose 
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expression. The suggestion just made consequently applies to the 
reading of a sequence of themes as much as or more than it does 
to the reading of a single one. 

Often instructors who inquire closely into the psychological 
bases for imagery, metaphor, and other verbalizations feel tempted 
to regard a piece of writing as almost a set of data for a case history 
rather than as a series of expressions. Although such an inquiry 
may yield valuable insights into the origins of difficulties and the 
sources of achievement, it should be remembered that such in- 
sights are valuable to the student of communication only if they 
can help him improve his writing. 

6. Attention to details.— It follows from the earlier suggestions 
in this section that the reading of a theme should be as painstaking 
and as attentive to detail as possible. Although the committee 
shares the current tendency to rate mechanical errors as not highly 
important in themselves, it does believe that all minor details, 
mechanical or othenvise, deserve critical examination. Such details 
may be significant in two ways: they may markedly interfere with 
or help the ■writing (a misspelling sometimes stumps the reader 
and a casual figure of speech may light up an entire subject), and 
they may be related to more distinguishing features of the com- 
munication, expressing a controling tendency or revealing a char- 
acteristic habit of thought. 

In practice., the theme reader must be selective in his notes on 
student papers because he is pressed by time and because he must 
not confuse his students. After a full reading of the theme, he will 
probably wish to focus his commentary on those elements which 
show interconnections and relationships, and ■which are parts of 
the major patterns of the theme. 

7. Reciprocal relations of discovery and judgment.— 'The reader 
starts to lay the groundwork of his concluding judgment at the 
beginning of his analysis. It is at first a mere impression— to be 
checked, amplified, and corrected by rereading. Although discovery 
of the qualities of the theme logically precedes the appraisal of it, 
the two activities make up an indivisible process; hence, a con- 
sistent separation between description and evaluation should not 
be expected. The vocabularies of these processes are to some extent 
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interchangeable, so that a description may be worded in the Ian 
guage of values and the statement that a certain thing has been 
done may imply a judgment 

The concluding judgment on a theme should be specifically re 
lated to Its subject matter and should clearly follow from the run 

“ ’“P'y evaluation 

1 ^ merely a mechanical summary of points already estab- 

lished rather by its consideration of the total performance it has 

mbunons'*^' perhaps invaluable con 

tinn being analyzed is a good one, there is a tempta 

on to limit the observations to minor flaws and conclude with a 
aZ rZ ^“Statulation to the student But a full and conscien 
The account of successes as well as failures 

™ reTv aT,r ™ should be nearly if not 

ment mu t m ” T “ “'“^'■*t=«ory, the general com 
idea of theme Z “''g^3''i>ate or condemn The 

on tins nroTect was'"® ‘he committee 

error with some stamhi' hT Y***'^*' "icrely fathoms the depths of 
adequacy of a uniform I instead challenges the 

as a total action and tn grasp the communication 

tion and ends of its subject Tauer 

a raranf b^whmh Tealhem ' intended as 

evaluation activities Used either f ‘h'= finality of their 

for staff discussion, the princiDles”F'’K"“^“^' “ bans 

companying themes could provide” analysis and the ac 

Houever, a rather na^tv n/* insights hitherto unrealized 
manded a thoughtful cmLmuo/rf principles de 

interacting parts The wnv #Ka. enure theme and its 

turned to students provides httle'o^n^^? typically graded and re 
from careful scrutiny of their worlTGen'^^fr 
assigned a grade, and returned to ^ ^"erally themes are read, 
hieroglyphics referring the student “oTraTeb 

more interested in the (trade thar. c ™ ^ h°nfi The student, 

seek to correct the tew mechanical emrs otL^n'’ 

Of spelling or use brought 



OBJECTIVES IN COMMUNICATIONS 91 

to his attention by the teacher. Rarely is the student required to 
reconsider his theme as a total product which has genuine strengths 
but also definite limitations. Using the Handbook for Theme An- 
alysis as an adjunct to teaching might prove a way of rectifying this 
condition. 

Several uniformly successful experiments were made using the 
Handbook as a teaching device. Students were given copies of the 
book and asked to study the principles. Then the themes them- 
selves and their criticisms were discussed as showing how some stu- 
dents had succeeded or failed with respect to their assignment. In 
another class, analyzed themes were extracted from the book and 
shown on a screen by means of an opaque projector. After a dis- 
cussion of the analyses, other unanalyzed themes were shown and 
the class was asked to make a similar critical analysis and judgment. 
In still another class, discussion of analyzed themes preceded asking 
members of the class to evaluate written work of their peers with 
equal thoroughness. All of these activities rested on the assumption 
that the act of criticism and judgment was positively related to 
increased ability to write effectively. In the experience of the users 
of the Handbook this assumption was warranted. 

Speech Analysis 

Comments from the initial users of the Handbook for Theme 
Analysis were so uniformly favorable that the Communications 
Committee decided to prepare a similar work dealing with speech 
analysis. Again the basis for the project was the critical incident 
rating scale. To test the validity of the scale for speech rating, a 
number of student speeches were recorded on wire and played back 
to the committee members. However, the committee members felt 
that it was unrealistic to rate such speeches without being able to 
see the speakers, for the total presence of an individual and of his 
audience are essential components of whatever the rater should 
judge. This belief led to the preparation of a sound motion picture 
film of students giving speeches under classroom conditions. 

One class of the third term of Written and Spoken English at 
Michigan State College was selected for the film. This class met 
several times in the college television studio to become acclimated 
to the apparatus and activity of such a place. During this trial 
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period students gave speeches before the television camera but no 
pictures were made Then at a time unknown to the students, 
kinescopic films were made of certain students giving different 
types of speeches Since the speakers were unaware that they were 
being photographed, the resulting ten filmed speeches had an air 
of realism not to be found in acted speech situations Five of these 
ten speeches were edited into a fifteen minute film with an intro 
ductory statement, prefatory remarks for each speech, and camera 
shots of the class responding to each speaker 
The plan called for the speeches to be transcribed in a booklet 
and each speech to be followed by a detailed analysis of the pres 
entation in terms of the particular assignment made It was be 
leve t at copies of the film and the companion volume of speech 
evaluation and analyzed speeches could be used m svays similar to 
those involving the Handbook for Theme Analys.s 
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of view was based on the belief that communication activities in 
general education should be more closely integrated and more in- 
timately related to student or adult life. The contention was ad- 
vanced that the speeches in the film and the themes were essentially 
no different from student performance in more traditional speech 
or rhetoric courses. 

To such remarks several answers are possible. It is true that the 
specific performances were fairly formal in their design and that 
viewed solely by themselves they represented nothing particularly 
new. To so judge them, however, would be to miss the quality 
which gave them validity as part of the thinking of general edu- 
cation. The committee members were seeking to find ways of im- 
proving teaching and evaluation of all aspects of communication. 
They attempted to do this by means of a single rating scale. When 
that technique did not prove as successful as they ivished, they 
attempted to accomplish the same ends by other means. As a 
matter of fact, at one time there was considerable discussion of 
preparing an entire battery of devices for measuring speaking, 
reading, listening, and writing-all in the context of the same 
subject. The scheme did not materialize because of a variety of 
factors. However, the intention illustrates that these efforts in the 
fields of speaking and writing were not discrete but were inter- 
dependent. Thus, the intentions and the approach place the theme 
and speech analysis activities squarely within the province of gen- 
eral education. 

In part, however, the contention is valid and must be accepted as 
just. In spite of course names implying a general education orienta- 
tion and in spite of a few exploratory ventures such as the critical 
incident approach to evaluation, fairly old wine of traditional 
speech or rhetoric is being offered in new bottles, or perhaps even 
in some old bottles with new labels. ^The facility with which the 
separate theme and speech projects were accepted evidences this 
tie with tradition. Particularly does this become apparent as one 
visits communications courses. With almost no exception, much of 
what is being done with students in respect to writing could have 
been observed in college classes fifty years ago. Apparently the 
integrating factor for the communication skills has either not been 
found or is not yet being implemented, and this needs major study. 
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Perhaps emphasis is too much on the process of communication 
and not enough on the preparation for it One possible integrating 
factor-the thought process, about which we shall say more m 
chapter x_is largely ignored in preoccupation with the character 
istics of the process 
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The several items given below are illustrative of the kind found 
in the test. 

1. Who is represented as the speaker in Passage III? 

1. Lincoln 

2. Jesus 

3. Thomas Aquinas 

4. Homer 

2. Which of the following descriptions of mattes role in life as 
conceived in these passages is most accurate? 

1. I: Man is a tool-usinganimal. 

2- II: Man is the master of his faie. 

3. Ill: Man is a child of God. 

3. Which passage agrees with the thought expressed in the follow- 
ing quotation? 

“Life’s but a walking shadow, a poor player 
That struts and frets his hour upon the stage. 

And then is heard no more; it is a tale 
Told by an idiot, full of sound and fury. 

Signifying nothing.” 

1. Fassagel 

2. Passage II 

3. Passage IH 

4. None of them 

These items clearly indicate that the test demanded a high de^ 
gree of reading ability plus an equally high level of analytical 
thinking. They also refute by example the often-made contention 
that objective-type test questions can only measure knowledge. 

The second part of the test consists of a reproduction of a student 
analysis of the preceding three passages and questions concerning 
the way in which the theme was written. This portion of the test 
was designed to measure writing ability by means of such ques- 
tions as: 

In items to , assume that the student’s purpose was to 

show that success in work is important. 

1. In the light of this purpose, to show that success in work is 
portant. his review of the passage is 

1. adequate, for he covers their chief objections to regard- 
ing success in work as important. 

2. adequate, for he points out that/* Jy fundamental 
objection is to 
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3 inadequate, for he includes only what is relevant to his 
purpose 

4 inadequate for he neither recognizes nor refutes impor 
tant objections 

2 Lines to Which is the most accurate interpretation 

of what Passage I meant? 

1 People are often hurt 

2 Workmen are injured 

3 Investors are defrauded 

4 Lives are used up 

Research Data from the Test of Critical Analysis 
IN Reading and Writing 
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data suggested that the test was much too difficult for freshman stu 
dents Regardless of the capabilities of students taking the test 
their scores were extremely low and somewhat less related to level 
of academic aptitude than were other Study tests From these data 
members of the committee reached the conclusion that perhaps 
the test was so difficult that students would give up without trying 
on the test In one situation, for example, a group of students was 
administered a four hour battery of tests with the Critical Analysis 
in Reading and Writing Test given last The same battery was 
given to a similar group of students with the exception that the 
Critical Analysis Test was given first The group taking the Critical 
Analysis Test last demonstrated a marked decline in morale, and 
a number of students failed to finish the test The group taking it 
first reacted differently, possibly because they took the test when 
they were considerably fresher, but there can be little doubt that 
many students are discouraged by the unfamihanty and difficulty 
of the tasks posed for the students 

This somewhat pessimistic view toward the potentialities of the 
test caused a number of individuals to refrain from carrying on 
studies designed to show how ability in cntical analysis might be 
affected by college classes A few persons, however, either because 
of curiosity or need to employ the test, regardless of its capabilities 
made further studies which, to some extent, controverted the first 
conc?usfon A doctoral candidate wTorfr/ng- with the Critical Analy 
SIS Test in Reading and Writing, the Cntical Thinking Test, and 
the Inventory of Beliefs demonstrated that over a year period 
differences between pre and post test means of a large group of 
freshmen were highly significant at the 1 percent level of confi 
dence ® He further demonstrated by analysis of variance of gains 
that students’ academic preferences as expressed in their selected 
major were significantly related to differences between gams again 
at the 1 percent level of confidence 

The Evaluation Office at another institution administered the 
Critical Analysis Test to matched groups of beginning freshmen 
beginning sophomores, and seniors, and reached the conclusion 

*Virtor Horomcr A Study of Selected Interest Factors as Related to Outcomes 
of the Program of General Education at Michigan State College (Unpublished 
doctoral dissertation Michigan State College 1954) 
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that there were statistically significant gains made on this test in 
the freshman year and again sometime between the end of the 
freshman year and the senior year Since these were matched cases 
on the basis of academic aptitude rather than identical cases tested 
and retested with the same instrument, there could be no practice 
effect Still further data were obtained Identical students at one 
institution were tested at the beginning of the freshman year, the 
end of the freshman year, and the end of the sophomore year The 
tabulation below indicates the gradual nse in these students* scores, 
ince t ese students took their formal training m communications 
during the first year, the gams recorded for their sophomore year 
are understandable ^ ^ 

Pre freshmen 
Post freshmen 
Post sophomore 

/o 18 88 4 45 
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In view of this failure, and in view of . 1 , 
date of the Cooperative Study, the comi^t terminal 

/. e committee was faced with the 
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alternatives of either dropping the objective test project or revis- 
ing either the original or the nexver form of the test. After careful 
deliberation, the committee elected to modify the original test 
because, in the opinion of the members, this test best exemplified 
the sophisticated kind of reading ability which they hoped to de- 
velop in college students, although they quickly agreed that at the 


TABLE 14 

Mean Gains or Stutents on CnmcAL Anaetbis in Beadino and 'Writino Test, 
CLABsmo) AccoBDota to PnE^TrsT Standwo 



present time few students are given the kinds of experiences to 
qualify them to do this sort of reading. As has been indicated 
earlier, the test was again revised by changing the order of some 
of the items and replacing some of the more difficult items mth 
somewhat easier ones, and lengthening the test by ten items. 
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there is no further need for analysis here except to underscore the 
point that Institution 5, whose students showed the highest pos 
sible gam, was an institution having an administratively inde 
pendent general education program required of all students This 
may be a major part of the explanation of the greater gam for 
these students Another possible explanation, however, might be 
that that institution also maintains a full time board of examiners 
who prepare tests of a nature similar to that of the Critical Analysis 
m Reading and Writing Test which are taken by all students in 
the general education program It may be that familianty with 
this type of testing could account for the differential gams made 
by the students from that institution 
To show how the Test of Critical Analysis in Reading and 
Writing related to several other tests of the Cooperative Study, 
Table 15 below is presented Perhaps the most significant factor 
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true we are not prepared to answer, other than that it may be 
partly due to the lower reliability o£ the reading and writing test. 

Summary and Implications 

This chapter began by sketching the crucial problem facing 
communication people. Can communication be conceived of, 
taught, and evaluated as an entity, or must the separate skills con- 
tinue to be regarded as discrete? The problem still exists, although 
the experiences of the committee may have shed some light on it, 
even if only to underscore its complexity. 

The critical incident categories revealed some of the dimensions 
of communication as illustrated by what students actually did. The 
theme analysis principles revealed more of what teachers might 
look for in judging themes. The descriptions of the traits pur- 
portedly measured by various institutional tests added to the com- 
posite. Finally the specifications of the Test of Critical Analysis in 
Reading and Writing pointed to other facets of the communicative 
act. 

As we lay these various aspects side by side, a pattern begins to 
emerge which, although it requires much empirical validation, 
may well be the integrative factor necessary to link all phases of 
communication into a meaningful whole. The basic element of 
communication appears to be the ability to think critically. The 
piroosss of reading, writing, speaking, or hsteningcan be described 
as the process of determining the existence and nature of a prob- 
lem, the posing of hypotheses concerning it, and the testing of these 
hypotheses by various means to arrive at a conclusion. Cardinal 
Newman told us long ago that "Thought and speech are insepa- 
rable from each other. Matter and expression are parts of one; style 
is a thinking into language.” Curricula designed to teach effective 
communication must first advance the idea that the essential need 
is the solving of problems— to divine what an author’s meaning is 
—to clarify one’s thought so that a problem stands clear in one’s 
own mind prior to being communicated— to listen to what is rele- 
vant and to discard the chaff. By acceptance of such a view, differ- 
ences among speech and composition teachers may be resolved. 
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there is no further need for analysis here except to underscore the 
point that Institution 5, whose students showed the highest pos 
sible gam, was an institution having an administratively mde 
pendent general education program required of all students This 
may be a major part of the explanation of the greater gam for 
these students Another possible explanation, however, might be 
that that institution also maintains a full time board of examiners 
who prepare tests of a nature similar to that of the Critical Analysis 
m Reading and Writing Test which are taken by all students in 
the general education program It may be that familiarity with 
this type of testing could account for the differential gains made 
by the students from that institution 
To show how the Test of Critical Analysis in Reading and 
Writing related to several other tests of the Cooperative Study, 
Table 15 below is presented Perhaps the most significant factor 
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ance of this very general statement. Some of the issues raised were: 

(1) Should not the phrase "knowledge of the nature and structure 
of science" be added to the objective? (It was not added on the 
grounds that this is implied by understanding and application, and 
that even appreciation at any level other than ingenuous fascina- 
tion presupposes some knowledge.) (2) Does the objective give 
enough emphasis to the impact of science on our culture? (This 
svas resolved as being a matter of interpretation of the objective.) 
(3) In view of this concern about the implications or the impact of 
science, a concern which also involves the social sciences, should 
not other than scientists be involved in working on this objective? 
(This was considered highly desirable and an attempt xvas made to 
arrange for such participation.) (4) What areas of science should be 
represented in the committee membership and would one commit- 
tee suffice? 

Perhaps the most critical question was that of the number of 
committees necessary. Personal visits of the Director with prospec- 
tive committee members elicited some strong arguments for txvo 
separate ventures, the key point being that physical and biological 
science courses are commonly offered separately and by distinct 
teaching staffs. However, a majority felt that a composite venture 
should be attempted, since the objective of immediate concern 
transcended the usual course divisions. Also in favor of the latter 
plan was the expressed hope that the project might do something 
to encourage the interaction of all general education teachers, 
regardless of their instructional area. After the first meeting the 
sentiment was clearly for a unified science committee. Although 
the physical and biological science teachers occasionally divided 
for work on special tests, they did so as subcommittees to carry out 
a task decided upon by the larger committee. Not only did the two 
groups work well together, but there was even clear indication that 
this experience encouraged a considerable degree of cooperation 
and commonality in thinking at the local level on several campuses. 

The generality of the objective accepted at the beginning of the 
project has been noted. The following more specific problems were 
suggested by the committee early in the first session: 

1. Evaluation of the ability to apply science knowledge to new prob- 
lems and situations. 
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Intercollege Committee on Science Objectives Nelly Bosma, 
Wright Junior College, Charles F Byers, University o£ Florida, 
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College WiLUAM H Matthews, Kansas State Teachers College 
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Stephens College, A L Vaughan, University 
Minnesota Earl K Wallace, Pennsylvania College for 
SScollege°“' Agricultural and Me- 
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current science materials is an important outcome of a general 
education science course. Even so, the ability in itself is insufficient; 
unless the ability is used, nothing of permanent value remains. 
The individual who does read and evaluate science materials is 
constantly enhancing his education and he is also able to function 
more intelligently as a citizen in the ever-expanding sphere of 
interaction of science and other disciplines. 

The viewpoint just stated was not immediately acceptable to a 
members of the Science Committee. Certainly it never became 
acceptable to all the science faculties in the cooperating colleges. 
Visits to campuses and discussions with individuals and groups 
involved in the teaching of science in the general education cur- 
riculum revealed some indifference and occasionally even active 
antipathy to such an objective. One typical negative comment was 
to the effect that with the time available there exists no alternative 
to placing emphasis on coverage and, even so, so many irapor an 
aspects of science are omitted as to call into question the possibility 
of achieving a respectable grasp of science. Another type o co 
ment was to the effect that such outcomes as those discussed above 
are difficult to achieve, even with majors and graduate 
and probably impossible with freshmen and sop omores. e 
to this contention is the agreement that given sufficient 
and understanding of science, the application of it is dependen 
the individual. Associated with this is the viewpoint t at it may 
a concern but not a responsibility of the instructor t 
continue to expand their knowledge and to apply it- e 

pessimistic and even disdainful thought expressed a out a gene 

education science course (by a teacher of such a course) 
students would be more gullible in their science rea mg e 
they would have a casual but inadequate acquaintance wi 
and concepts, and a consequent tendency to be impresse ^ 
statement using them. On the other side, it ^ 

the very real possibility of increased gullibility wou 
some active attention to current science materia s w ic 
counteract that effect. 

The Problem of Content 

In discussing the problems of evaluation and instruction, att 
- tion had to be given to the content of the science courses, n 
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2 Evaluation of the ability to analyze scientific data summarized in 
maps, tables, curves, charts, graphs 

3 Evaluation of the ability to read and evaluate news articles and 
popular writing on scientific developments 

4 Evaluation of the understanding of the role— importance and 
limitations— of science in the modern world 

5 Evaluation of the willingness to face facts, to revise judgments, and 
to change behavior in the light of appropriate evidence 

6 Evaluation of the understanding of the point of view with ^\hich a 
scientist approaches his problems and of the kinds of things that he 
does 

7 Evaluation of the recognition of the need for additional saence 
knowledge in a situation, and the ability to acquire it 

The first, third, and sixth of these problems were of greatest 
interest and, in fact, were accepted as including most of the ideas 
involved in the remaining four There was general agreement that 
a single project was more desirable than three, and it was also rec 
ognized that the three were not entirely independent A happy 
solution was reached by accepting news articles and popular 
writing on science as the source of materials for testing both the 
ability to apply science knowledge and the ability to demonstrate 
understanding of the scientific point of view Clearly, the reading 
and evaluation of current science material must involve the other 
two abilities as well as others, although the full appreciation of the 
interrelationship of these abilities is a continually maturing con 


Importance of the Ability To Read Science Articles 

The case for the emphasis thus placed on reading current news 
ar 1C es and popular (and semipopular) writing about science in 
volves three factors (1) Science ,s constantly Lvelopmg. so that 
nfVmi reading may soon find his knowledge 

19 \ scientific developments virtually nonexistent 

tn ihf* ”iany attempts to bring science and pseudo science 

accurarv^a'^T barrage of various degrees of 

(3) For the individual whose tornial 
rXf science is limited to one or two general edu 

Xnah science wilf be largely 

From these considerations, „ follows that the ability to read 
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will continually find that he is confronted with the necessity of 
increasing his store of knowledge in order to understand a nciv 
scientific development. Lackingany knoivledge of science, an indi- 
vidual cannot read intelligently about science. Having some knowl- 
edge of the field, some grasp of the methods and viewpoint of sci- 
ence, and a concern about the implications of science for the pres- 
ent and the future of civilization, it is at least possible that he may 
continue contact with science with some satisfaction to himself. 
These considerations provided at once a justification for the im- 
portance of the objective and a detachment of it from the problem 
of content selection and testing. 

Further Elaboration of the Selected Objectives 
As a first step, members of the committee undertook to examine 
such sources of materials as (1) science sections of Time, Newsweek, 
New York Times, Science Digest, Reader's Digest, and the like; 
(2) journals such as Scientific Monthly, Scientific American, Sci- 
ence, American Scientist; (3) government and industrial bulletins; 
(4) popular science books, science fontasies; (5) text and reference 
books, histories of science, enc)’clopedias. The purpose was that of 
locating cases or excerpts which would typify the kind of material 
which teachers felt that students should be able to handle and 
which might serve as a basis for the development of a series of 
questions. The decision to include texts, reference books, and the 
like resulted from a decision to broaden the scope of possible ma- 
terials, in view of the fact that timeliness or relevancy is not always 
a matter of being current. Coincidental with this, tests and test 
items were reviewed to determine the availability of items meeting 
the requirements of the specified objectives. Many of the excerpts 
collected were given to students to read and comment upon or 
^vrite about. Likewise some of the objective test items were given 
and the nature of student responses summarized. 

As the committee at its first workshop undertook to create test 
items, it became clear that the terminology involved in the ob- 
jectives was too vague to give much aid. The necessity for a careful 
and detailed analysis of the kinds of activities to be expected of 
students in relation to the three stated objectives was recognized. 
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whole however, the nature of the objectives chosen and the em 
phasis of this project permitted the issue of common content to be 
minimized The lack of reference to it must not be construed as a 
lack of concern or as a suggestion that students can learn to apply 
science or read science articles without knowing any science 
Rather, the justification for inattention to testing of content results 
from (1) a feeling that testing of science knowledge has already 
been extensively and reasonably satisfactorily dealt with in stand 
ardized tests, even if other such tests are needed, they are relatively 
easy to develop, (2) the fact that most colleges expressed themselves 
as being satisfied with the present testing of knowledge or were 
even concerned about the overemphasis accorded it, (3) the fact 
that the planned emphasis of the Cooperative Evaluation Study 
was to be on those general education objectives regarding which 
little effective evaluation had taken place, (4) the fact that the 
variety in subject matter in the various science courses would make 
It difficult to establish a common denominator for subject matter 
testing The lack of such a commonly acceptable body of subject 
matter was not regarded, however, as an obstacle for dealing with 
non subject matter objectives It does require that the problems 
pose or students contain a fairly complete statement of relevant 
intricate subject matter ramifications 
pni considerations is the assumption that, m gen 
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d) to recognize in a line of reasoning whether an observation plays 
the role of a premise or verifies the conclusion 

e) to recognize in a paragraph or statement proper or improper 
use of such reasoning as deduction, induction, ating authority, 
or analogy 

/) to differentiate fact and assumption in a paragraph or state 
ment 

g) to recognize the difference between statements or interpreta 
tions based on scientific evidence and those which contain 
opinion 

5 Attitudes— Siik. a question which requires the student 

а) to recognize in a paragraph or statement proper or improper 
use of such concepts as causality, teleology, simplicity, con 
sistency, tentative nature of truth, operationalism 

б) to assess a situation and recognize appropriate action in 
harmony with the nature of science and society 

The guide is, in a sense, a definition of the objectives in terms of 
behavior expected or desired from students As such it has several 
uses First, an article or extract thereof may be studied to deter 
mine to what extent it provides opportunities to raise questions 
possibly evocative of one or more of these behaviors Although not 
quite envisaged as such, the guide thereby becomes a means of 
evaluating materials from the viewpoint of their significance for 
testing Long reading passages providing opportunities for few 
questions are hardly justified Materials which include too much 
information or discuss very exphcitly points upon which questions 
might be raised must either be revised or discarded Second, the 
guide was found to have implications for teaching In preparation 
for a class, some teachers found it helpful to check trough the 
guide with the purpose of planning how the day s work might be 
organized to involve the maximum of behavior of the type speci 
fied Perhaps at this point there came an almost unanimous reac 
tion that most of the science courses represented in the committee 
provided little opportunity for such behavior Next, the decision 
was reached to attempt to deal simultaneously with both instruc 
tional and evaluation problems, not however without some dis 
senting votes from a few who felt that evaluation and research— 
rather than instructional methods— were the chaise of the com- 
mittee Evaluation continued as the central concern, but with a 
continual awareness of tlie instructional implications 
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Members of the committee, with the able assistance of Dr Louis 
Hell, the consultant, prepared such an analysis m outline form 
The statement underwent a number of revisions and finally as 
sumed the following form 


GUIDE FOR CONSTRUCTION OF 
EVALUATION ITEMS 

1 Problems — a question which requires the student 

<z) to identify the problem to which the statement gives the answer 
and to recognize the central problem to which a number of 
statements are addressed 

6) to indicate with reasons, whether or not a given problem is 
stated specifically enough to begin an investigation of it to ob 
tain an answer 

e) to indicate whether certain nonscientific factors, such as value 
judgments or matters of faith, are contained in the problem 
thereby making a scientific solution impossible 

2 7n/ormafion (data, laws, and prinaples)— ask a question which re 
quires the student 

a) to recognize when the information he possesses is inadequate 
for a given problem 

b) to indicate kinds of sources of information appropriate for a 
given problem 

c) to evaluate the authentiaty of given sources of information in 
relation to a given problem 

d) to indicate his ability to apply information he possesses or has 

^ T 4 ^ solution of a given problem 

Hypotheses-^ik a question which requires the student 

) o ormu ate or recognize hypotheses based on given data or 
situations ® 

evidence necessary to judge the truth of a given 
deduction from a hypothesis 

hypo™esir^ expenment which will test the truth of a given 

^ or^dn°nnr^^ When observauons or experimental data support 

e) to rero!»ni 7 ^^/°^^ ^ g»ven hypothesis and to what degree 

hypothesis involved in the formulation of a 

^ r°eco™it 

or condnsion ' involved in an interpretation 

c) to involved in a conclusion 
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d) to recognize in a line of reasoning whether an observation plays 
the role of a premise or verifies the conclusion. 

e) to recognize in a paragraph or statement proper or improper 
use of such reasoning as deduction, induction, dting authority, 
or analogy. 

f) to differentiate fact and assumption in a paragraph or state- 
ment. 

g) to recognize the difference between statements or interpreta- 
tions based on scientific evidence and those which contain 
opinion. 

5. Attitudes— zsk a question which requires the student: 

a) to recognize in a paragraph or statement proper or improper 
use of such concepts as causality, teleology, simplicity, con- 
sistency, tentative nature of truth, operationalisra. 

b) to assess a situation and recognize appropriate action in 
harmony with the nature of science and society. 

The guide is, in a sense, a definition of the objectives in terms of 
behavior expected or desired from students. As such it has several 
uses. First, an article or extract thereof may be studied to deter- 
mine to what extent it provides opportunities to raise questions 
possibly evocative of one or more of these behaviors. Although not 
quite envisaged as such, the guide thereby becomes a means of 
evaluating materials from the viewpoint of their significance for 
testing. Long reading passages providing opportunities for few 
questions are hardly justified. Materials which include too much 
discuss rerj' expiicSily wpvcw whJcJi qaessSsxos 

might be raised must either be revised or discarded. Second, the 
guide was found to have implications for teaching. In preparation 
for a class, some teachers found it helpful to check through the 
guide with the purpose of planning how the day's work might be 
organized to involve the maximum of behavior of the type speci- 
fied. Perhaps at this point there came an almost unanimous reac- 
tion that most of the science courses represented in the committee 
provided little opportunity for such behavior. Next, the decision 
was reached to attempt to deal simultaneously with both instruc- 
tional and evaluation problems, not however without some dis- 
senting votes from a few ■who felt that evaluation and researdi— 
rather than instructional methods— were the charge of the com- 
mittee. Evaluation continued as tlie central concern, but with a 
continual awareness of tlie instructional implications. 



110 


GENERAL EDUCATION 


The guide was regarded as defining one axis of a chart or grid 
of specifications for the development of a test The other axis was 
to be a specification of subject matter or problem areas to be 
covered Here some difficulty arose in that there was a natural 
desire for broad coverage accompanied by a recognition that real 
coverage of the materials commonly dealt with in physical and bio 
logical science was truly impossible In a sense, the subject matter 
chosen was thought to be relatively unimportant since the abilities 
outlined in the guide could be equally well tested in many differ 
ent contexts Yet balance interest, and avoidance of bias suggested 
some delineation of subject matter Figure 4 illustrates one of the 
forms of the gnd on the basis of which early tentative forms of tests 
were constructed In this gnd the cells are filled in with names 


identifying particular passages 

It was found impossible to cover the complete range of subject 
matter or problems suggested by this grid An article sufficiently 
involved to permit testing any one of the abilities usually per 
mitted testing of others, and economy of time in the sense of the 
relation of the time spent in reading to the time spent m response 
dictated taking full advantage of that fact In other words, if ten to 
twelve Items are posed on a single reading selection it is clear that 
four or five selections is about the limit for a Hfty minute test 
Difficulties in communicating to science teachers not directly 
invo ve in the committee work indicated two further steps in 
c ^1 ying the abilities Illustrative items involving each of the 
su points of the guide were prepared, and several teaching units 
were also prepared indicating how the abilities might be involved 
m classroom activities Some of the latter are to be found in the 
Reasoning and Understanding^ The prepara 
Xi°v f It clear that items which could be 

reoresemeH 

tLs arise f <>£ the guide were difficult to find 

mmrrcbted aT.h *emselves are highly 

somewhafdrfterend; ='‘‘-ck a given task 


*Scietu:e Reuontng and Vnderstandme 
buque Iowa Vfm C Brown Co 1954) 


Handbook for College Teachers (Du 



J. Human Behavior 


I. Applied Science 


H. Growth, Develop- 
ment, and Inter- 
relationships of 
Living Or^nisms 



•Grid showing the skills and problern areas to be sampled in the Test of Science Reasoning and Understanding 
































112 


GENERAL EDUCATION 


The result of this was a reduction for practical purposes of the 
many specific abilities of the guide to the five noted on the grid. 
Even these were hard to distinguish at times, but the attempt to 
frame items dealing with five abilities or with the subpoints 
under them secures a better balance than would otherwise be 
the case. 

The Scientific Point of View 
One other development in the attempt to make the three origi- 
nal abilities more specific ^vas almost entirely the work of Dr. 
William C. VanDeventer, then at Stephens College, who was par- 
ticularly concerned with the characteristics of or the assumptions 
implicit in the scientific point of view. While this list was not 
directly used in the development of evaluation materials, it did 
influence considerably the interpretation placed on the third ob- 
jective, The list follows. 


BASIC ASSITMPTIONS OF A SCIENTIST 
1. Principle of Objectivity: 

A scientist cultivates the ability to examine facts and suspend 
judgment with regard to his observations, conclusions, and 
activities. 


2. Principle of Consistency: 

A sdentist assumes that the behavior of the universe is not 
capricious, but is describable in terms of consistent laws, such 
that when two sets of conditions are the same, the same conse- 
quences may be expected. 

3. Principle of Tentativeness: 

A scientist does not reprd his generalizations as final, but is 
o modify them if they are contradicted by new evidence. 

4. Principle of Causality: 

believes that every phenomenon results from a dis- 


5. Principle of Uniformity: 

believes that the forras which are now operating in 

trld and'Jh and that the 

continnrafs “V'™' '''>■'* see are the result of their 
continuous operation. 

6. Principle of Simplicity: 

widely applicable explanations of 

simnip tprm reduce his view of the world to as 

simple terms as possible. 
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7. Principle of Materiality: 

A scientist prefers material and mechanical explanations of 
phenomena, rather than those which depend on nonmaterial and 
supernatural forces. 

8. Principle of Dynamism: 

A scientist expects nature to be dynamic rather than static, and 
to show variation and change. 

9. Principle of Relativeness: 

A scientist thinks of the world, and of things in it, as sets of 
relationships rather than as absolutes. 

10. Principle of Intergradation: 

A scientist thinks in terms of continua; he distrusts sharp 
boundary lines, and expects to find related classes of natural 
phenomena grading imperceptibly into one another. 

11. Principle of Practicality: 

A scientist expects that in any situation involving competition 
among units of varying potentialities, those which work best 
under existing circumstances will tend to survive and perpetuate 
themselves. 

12. Principle of Continuous Discovery: 

A scientist hopes that it will be possible to go on learning more 
and more about the material world and the material universe of 
which it is a part, until eventually all may be understood. 

13. Principle of Complementarity: 

A scientist attempts to incorporate all phenomena into a single, 
cc»jsistent, natvra} sdieme, hvt he recognizes that contradictor}' 
generalizations may be necessary to describe different aspects 
of certain things as they appear to us. 

14. Principle of Social Limitation: 

The social framework within which a scientist operates may de- 
termine and limit the kinds of problems on which he works, and 
the data which he collects, and may also influence his conclusions. 

This list was developed by taking note of the various kinds of 
decisions made by scientists and attempting to identify the assump- 
tions or principles on which these decisions are reached. The list 
is regarded by VanDeventer as tentative and probably incomplete. 
In its present form an interesting assignment for students can be 
developed by asking them to read scientific articles, historical cases, 
and the like and identify remarks or conclusions which involve or 
illustrate these principles. Although there will not be unanimity 
on the association of specific principles witli particular passages, 
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the experience is a significant one in helping students to under- 
stand the point of view of a scientist. 

The Development of Evaluation Instruments 
Coincidental with the clarification of objectives, articles and 
extracts collected from many sources were used to formulate 
sample test items, and other faculty members 'were asked to and did 
contribute similar types of items. Most of these were reviewed and 
screened by a local committee or an individual on each campus. 
These were further screened, revised, and collected into trial forms 
of tests which were returned to the various campuses for adminis- 
tration to students and for collection of student reactions by oral 
or written comments. At this stage there was little concern that the 
trial fo^s be balanced tests, the emphasis being almost entirely 
on aving sufficient data on the items to provide some objective 
evidence on difficulty and discrimination. Based on the experience 
of administering these trial forms to students, the following speci- 
fications were drawn up for the prospective tests: 

1 passages used should be current and/or of interest 

Tr momentary interest as to date the test. 

^ test dealing with current writings 
9 v ^ more than two to four years without revision. 

genemWy involve between 400 and 

ineahac^T * rich as to be capable of provid- 

mg a basis for approximately ten items per passage 

test can h number of test items of the type used for this 

4 Fnr “‘"g ‘he rate oE one item per minute, 

in leneth hm^ the test should be about two hours 

class periodl ^ longer than fiEty minutes (the usual 

Stive LllT ' Science Committee chose the 

might be used d ^ ° j'."® Eorms so that one or both 

•- 

frot hetsTeanirir'^'^'u'’'’"''^ ^y test items ranging 
the paragraph to 

ciples needed for the solution to®. he prob,rm^' 

Out of the mass oE items, together with the accumulated sub- 
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jective reaction of faculty and students and the statistical data from 
tryouts. Forms A and B of a Test of Science Reasoning and Under 
standing (Natural Sciences) were put together These tests involved 
fifty Items each and were based upon five or six reading passages 
These forms were extensively used in the various colleges in the 
fall of 1951 and to some extent in testing the same students as soph 
omores in spring 1953 Data from these testings will be discussed 
in a later section Revised Forms C and D were developed in the 
last year of the project Tfiere is not as much data available on 
them as on the earlier forms, yet those data which were collected 
indicated the superiority of Forms C and D and resulted in the dis 
continuance of Forms A and B 

The makeup of Form C of the Test of Science Reasoning and 
Understanding is described by the grid shown in Figure 5 In this 
grid four points have been assigned to each of the fifty items and 
the points for each item have been divided among the five abilities 
on the basis of a committee judgment The items dealing with the 
article on Chubb Crater are good examples of the abilities tested 
The article itself is given first as in the test and then five of the 
nine items on the article are given with an indication of the ability 
tested, using the Roman numeral shown on the grid Items 47, 48, 
49, and 50 are not related to the article but are included as indi 
cations of the attempts made to get at the scientific point of view 


Illustrative Examples 

Items 38—46 are related to the following selection 

Read the following selection carefully It is adapted from Time 

The northwestern tip of Quebec just south of Baffin Island is 
flat, sodden tundra sprinkled thickly with little lakes Most of them 
are irregularly shaped But Prospector Fred W Chubb noticed, 
while poring over an aenal photograph, that one Jake was almost 
round and surrounded by a wall of rock This week Dr V B 
Meen, field geologist, Ontario Department of Mines returned 
from a quick air visit to the lake and reported that it was almost 
certainly a meteorite crater (there was no lava or other sign of 
volcanic activity) and the biggest yet discovered The lake in the 
crater is two and one half miles across, compared with Arizona's 
famed meteorite crater, which is four fifths of a mile across Its 
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level is about 80 feet above that of other small lakes in the vidnity, 
and around it is a ring of shattered granite that rises 550 feet above 
the tundra. The rim is lowest on the northwest side, which suggests 
that the meteorite came from that direction and hit the ground 
obliquely. 

Dr. Meen found no meteoric iron, only a reddish rock that might 
prove to be the peculiar stony material of which some meteorites 
are made. But there was plenty of other evidence that some 
enormous body had buried itself in the earth: shattered blocks 
of stone from football to freightcar size, and concentric circles in 
the granite around the crater, like ripples stirred up by a pebble 
dropped into still water. Dr. Meen estimated that the meteorite 
must have fallen at least 3,000 years ago, since there are no Indian 
or Eskimo legends about it. He named it Chubb Crater after the 
sharp-eyed prospector, and promised that a full-dress expedition 
would report on it within a year. 

Vire^iwns: For each of the items 38—46 select the best answer, 
then mark the corresponding space on the answer sheet. 

38. The basic problem which faces a scientist in the case of this 
lake is the 

1. shape of the lake. 

2. size of the lake. 

3. high elevation of the lake. 

4. origin of the lake. 

5. depth of the lake. 

(Ability 1) 

40, Which one of the following statements is l^ast related to the 
basic problem? 

1. “The northwestern tip of Quebec is flat, sodden tundra, 
sprinkled thickly with little lakes.” 

2. . . one lake was almost round, and surrounded by a 
wall of rock.” 

3. “There was no lava or other sign of volcanic activity.” 

4. “Its level is about 80 feet above that of other small lakes 
in the vicinity, and around it is a ring of shattered 
granite that rises 550 feet above the tundra.” 

5. (There were) “concentric circles in the granite around 
the crater, like ripples stirred up by a pebble dropped 
into still water.” 

(Ability 11) 

43. Whicli one of the following lines of study would probably 
furnish the most conclusive means of testing the idea of 
meteoric origin of the lake? 



118 


GENERAL EDUCATION 


1 Compare the lake with other lakes in the immediate 
vicinity 

2 Accurately survey the lake and the territory immedi 
ately around it 

3 Look for other lakes like this lake in other parts of 
Quebec and Balfin Island 

4 Compare the lake in detail with known meteoric era 
ters 

5 Determine the structure and composition of the rock 
found in the vicinity of the lake 

(Ability III) 

46 According to some theories of the ongin of the solar system, 
the earth and moon and other planets and their moons were 
^rmed at one sta^ by a process m ivhich the larger frag 
thT smaller ones as they all traveled^around 

tes occurrence of meteor 

men^wh^.h ? ^ bombard 

mem which took place in earlier days Following out this 

^ it^mly h.sw^'^ bombarded as the earth was m 

' ?omrrdme„r'““ 

mMn°is notTh in the formation of the 

4 Te chinS^^^ ?■"' *'= f°™at.on of the earth 

comtTa 'he moon are 

5 large meteo'r.S, sm ,Llar"th' 

uninhabited places * “"‘j' ■" 

(Abilities III and IV) 

According to your understandme of th^ « c 

mark space * * scientific point of view, 

1- if the statement is warranted 

2- .f the statement ts ,n confl.ct wtth the saenttfic po.nt of 
with the sc”ntife,^'n,'''£ 

clittalm SrcarefTp^Sd’4T“'“‘ 

48 A scientist reads carefully the resnit. t .n 
reported by other saenttsB so that he carf n*'" 
ment that avoids some of their errors” 
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49. A scientist thinks of the laws of nature as decreed by a divine 
intelligence. 

50. A scientist assumes that scientific laws are no more than de- 
scriptions of relationships in natural processes. 

Item 47 might be regarded as involving the Principles of Objec- 
tivity and Tentativeness; item 48, the Principle of Continuous 
Discovery; item 49, the Principles of Materiality and Causality; 
and item 50, the Principles of Consistency, Causality, and Rela- 
tiveness. 


The Physical and Biological Science Tests 
In addition to the natural science forms of the Test of Science 
Reasoning and Understanding, a physical science and a biological 
science form were also prepared. These differ from the natural 
science form in two ways; first, as the titles imply, the material is 
restricted to either the physical or biological science and, second, 
the level of difficulty, primarily in terms of the amount of knoivl- 
edge of the field required, is greater. The abilities tested are the 
same and, in fact, many of the items used in the physical and bio- 
logical science forms are used also in the natural science form. The 
special forms are somewhat more appropriate for use in courses 
restricted to one area of science. One difficulty faced in making 
such tests is that the distinctions between biological and physical 
science are more easily maintained in the development of courses 
than in current articles, many of which report on research bridg- 
ing, either in purposes or in methods used, the gulf between the 
physical and the biological sciences. Despite the interest in and the 
effort put forth to develop the special forms of the tests, relatively 
little use of these was made by participants in the Study. Perhaps 
the best case for their use can be made in a situation where the 
teacher of a science course restricted to one of the science fields 
irishes to use a test to explore the abilities involved and feels that 
student motivation will be higher if the test is more specifically 
related to the course. It has been found that students exposed only 
to physical science and to little direct training in the abilities in- 
volved in these tests react against biological science articles as 
“having no relationship to the course.” 
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Concern about Non objective Materials 
We have noted that, very early in the project, members o£ the 
committee became concerned with the problem of teaching Only 
a person or two dissented from the point of view that to the extent 
that evaluation oriented activity suggested new techniques of in 
struction these should be explored and systematized This concern 
about instruction ultimately resulted in the development of the 
handbook entitled Science Reasoning and Understanding A 
Handbook for College Teachers, already mentioned This hand 
book describes the Guide for Construction of Evaluation Items,® 
indicates ways of using textbook material to teach to these ends 
and suggests some special teaching methods found valuable by 
members of the committee Other sections of the handbook discuss 
sources and the selection and modification of current materials for 
testing and instructional purposes The use of essays in both testing 
and m instruction is discussed at length m one section The utility 
and the purposes of essay testing were of particular concern to the 
committee, first because some of the weaknesses of objective test 
mg were fully appreciated and, second, because there uas an aware 
ness that complete preoccupation with objective materials would 
very likely prejudice others against the work of the committee 
The committee was concerned with essay materials in several ways 

1 Asa testing device No direct atuck was made on this problem 
but examples of superior practices were collected There are 
also numerous implications for essay testing resulting from the 
attempts to validate objective test items by essay questions 

2 As a teaching device The handbook, Science Reasoning and 
Understanding deals with this problem 

3 As a means o/ developing objective test items This use has 
a ready been noted m mentioning the possibility o£ written 
responses being requested to the stem of an item for which no 
alternative responses are listed 


4 means of validating objective testing procedures This wlf 

be discussed as a portion of the evidence on the validity of thi 
Test of Science Reasoning and Understanding 
•See pp 108-9 
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The Survey of the Use of Essay Materials 

The committee undertook to survey science teachers to see in 
what ways essay material was actually used in science classes. In ad- 
dition to the routine and well-known procedures, the following 
additional uses were suggested: 

1. Have the student read a description as to how one scientific prob- 
lem was solved and give examples o£ inductive steps, deductive 
steps, and hypotheses that were formed. The student is asked to 
make comparisons with other problems studied both as to purpose 
and general method. He is asked to give his opinion as to what 
would have happened to the experiment in its early stages had 
certain errors crept into the investigation. 

2. The student is asked to write an essay in which he is to present a 
convincing argument supporting the validity of a certain hy- 
pothesis. 

3. Another technique involves allowing cooperative effort on the 
part oE students in writing an examination. The essay question is 
presented in the laboratory. Two students, four students, or the 
whole class may collaborate in arriving at an answer to the ques- 
tion. It is possible that a minority answer may be presented. Each 
individual 'tsrrites up his own answer after the statement of the 
answer has been agreed upon by the group. The ideal group for 
this kind of examination consists of four students. 

4. Essay tests are used to motivate students to cut through a lot o£ 
subject matter in order to crystallize the point of a problem, a 
controversy, or a process. Essays are sometimes read in class. This 
is done to bring out such matters as inaccuracies of analysis and 
thought, hazy and inadequate statements that do not come to 
grips with the point at issue, or questions that are implied but 
left unstated. Essay tests are better used for these purposes than 
simply to cover particular topics, chapters, and pages in a book. 

5. Students are asked to write extensive papers, one at the end of the 
first semester, one about midway through their college career, and 
one just before graduation on topics that necessitate the drawing 
of information from all of the areas of general education. These 
papers are then graded by representatives of the various areas. 

6. A daily report is asked for, written on the form outlined below; 

Name 

Date of Class Experience 

Summarhe the class experience briefly: 

(pne-half page) 

Evaluate the dass experiCTice in terms of your own thinking. 
(pne-half page plus back of sheet) 
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The class experience reports are collected daily from the members 
of the group They are scored and returned to the students 

7 The student is required to read an article Later he is presented 
with several paragraphs from the same article and a quiz that has 
been formulated on these paragraphs based on the guide Each 
Item of the quiz is so constructed that an answer can be given in 
one or two complete sentences and only that much room is 
allotted for the answer 

8 A topic that IS considered “top news” in the natural science area 
is kept before the group This is done all through a semester, not 
necessarily as an assignment but as material for discussion Near 
the end of the semester each student is supplied with a summary of 
the articles from which material was drawn for discussion The 
students are allowed several days to study the material Then they 
are given an essay type examination built around the following 

^ to recognize and state problems, ability to select, 

to riro^n ^formation in relation to problems, ability 

to recognize, formulate, and evaluate conclusions 

Reasoning and Understanding, includes 

include her r material too lengthy to 

ess vanes o' n't" ^““"ons tndicate one form of 

essay question used by a number of teachers 

verse 26 unknown'rfe^^^ reports that there exist m another uni 
the alphabetTz .^y-b^lreally denoted by the letters of 

n c Vr Tr’t Their relative weights are given below 




STUVWXYZ 
ZS 33 43 53 63 35 45 47 59 


- -- — stf ta 03 03 35 45 4/ 5y 

dilTerem groupslicrording classified into 6 

lies as follows ® chemical and physical proper 


Group I 
Group II 
Group III 
Group IV 
Group V 
Group VI 


Active gases 
Liquids 
Metal solids 
Transition solids 
Non metal solids 
Inert gases 

predict mighTbV£oTn??‘"Explam®h.e'''“ 
get your answer. ^ " die reasoning you use to 


A, C. F, M, Q 

B, E, L, P. V 
D. H. K, U 
G, J. O, T 

I, S, X 

N. R, W, Y, Z 


[2" space] 
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(b) What do you predict the weights of these new elements to be? 

[114" space] 

(c) Indicate the group to which each of these predicted elements 
might be assigned. 

[2" space] 

An £ssay Approach to Validation 

The following questions are from forms developed for valida* 
tion purposes but have possibilities for general usage. The selec- 
tion used in the first illustration (sec below) is one that was 
taken directly from the objective test. It tvould also be possible to 
ask for an essay on the passage without structuring the response by 
a series of questions, but considerably less will be elicited from 
most students. 

At a temperature of 0° Centigrade and standard barometric pres- 
sure there are about 27,000,000,000,000,000,000 molecules in one 
cubic centimeter of air. These molecules are in constant motion, 
and have an average velocity of about one-quarter of a mile per 
second. This is slightly greater than the speed of sound. Under 
these conditions the distance traveled by a molecule before it 
strikes another is eight millionths of a centimeter. 

In interstellar space a molecule is believed to travel about 50 bil- 
lion miles before it collides with another. Each molecule in space 
coilides mch saachec oa die cE about once a fear. 

Questions 

18. What is your opinion of the reasonableness of the numerical 
values given in the passage? What is the basis for your 
opinion? 

19. What kind of reasoning would a scientist use to arrive at 
the statements made in the passage? 

20. The numerical values given in the passage are too large or 
small to be measured directly. Does the scientist just assume 
they are true and let it go at that, or what? 

21. If the passage is taken as true, what can be deduced concern- 
ing the distance a molecule moves between collisions in 
comparison with its sire? 

22. Would you say that the region between the earth and the 
sun is a vacuum? Explain. 
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The following questions do not depend on the preceding selec 
tion 

23 Does the scientist think that laws of nature cause and/or 
govern natural processes? Explain 

24 What do you think of the statement, ‘ A scientist conceives 
his task to be that of finding laws that fit with his already 
established view of the world ? 

25 Does the scientist expect that a new natural law will emerge 
from each new experiment that he undertakes? Explain 

Research Aspects of the Project 

Reliability 

The reliability o£ the Forms C and D of the Test of Science 
Reasoning and Understanding was computed for entering fresh 
man groups in a number of colleges Reliabilities computed by the 
Ruder Richardson Formula No 20 were found consistently to be 
in t e range 80 to 85 Since the test was planned for group 
measurement rather than for individual use, these coefficients are 
Tnn Forms C and D were administered— requiring 

the r'f'Th* °C testing time-the estimated reliability for 

the total, by the Spearman Brown formula, is about 90 

brsK tind the revision of the tests on the 

hivh deirr'"" difficulty and discrimination ensures a 

boffi vafd ^ internal consistency in the test This is related to 

uellLTs inld"^ - ‘he quality of the 

simolv exnert th manual, we shall not repeat it, but 

about the test. ” ** indirectly reHected in other evidence 

Validity 

for''^t*'e"s 0fThisTvD'''''’'r‘‘^ validity 
sahCs p^sts^n"^^^ P™- >°Sical, or face 

steps involved in the defelopmeroE thTtelrrh ‘^"T 

selections used are those agreed UDon aVh ' materials or 

be able to read, the abilmes .J”" “ hemg ones students should 
group of teachers as being important aTl “P°” “ 

for vnhdity rests on thefe judgment burtunl ™''’'’ 

were earned on to reinforce them " 
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Three questions were raised: (1) Do the abilities selected really 
describe what competent individuals do in reading scientific 
articles? (2) Does the student who rates high on the ability as 
measured by the test actually read science articles extensively? (3) 
How is ability to respond on an objective test about an article 
related to ability to react orally or in essay form to the same or 
similar articles? 

The first question was deemed to have an affirmative answer 
simply because of the techniques used in developing the tests. The 
second question proceeds beyond the immediate matter of valida- 
tion of a test to the habits, interests, or attitudes possessed by or 
inculcated in the person. The concern with validity is with 
whether the test reveals the extent to which a person has the 
abilities rather than with his habit of using them. This is not a 
denial of the importance of such habits or of their relevance as 
general education outcomes, but the abilities do have to be learned 
before they can be used and become habitual. 

A considerable amount of evidence relevant to validity was col- 
lected by essay techniques. Directed \vritten questions of the type 
exemplified on page 123 were prepared for all the passages used in 
the objective test. These were arranged in groups which could be 
answered in one or two hours. The first approach using this pro- 
cedure involved: (1) giving the essay test followed by giving the 
objective test; (2) scoring the written test, each question being 
judged by two raters on a 0, 1, 2, 3 basis and total scores of the trvo 
raters checked; (3) scores on the written test were then correlated 
with the corresponding questions and with the total score on the 
objective test. 

The second approach, using free response essays (no questions 
posed), was to be treated essentially the same way, but called for 
somewhat more specific instructions in rating. In practice, these 
free responses by students were so abbreviated and crude that no 
one carried out this particular type of validation. Table 16 exhibits 
typical results showing the range of correlations found between 
the objective test and structured essay responses. In this table, A 
and B refer to the earlier forms of the objective test, while C, D, E, 
and F refer to the essay questions. Each set of these covered ap- 
proximately one-half of the items found in tlie respective objective 



126 


GENERAL EDUCATION 


test, C and D being associated with Forms A and E, and F with 
Form B. In each case the essay scores are correlated with the total 
score on the objective test. 

These correlations are reasonably good for this type of material, 
and suggest considerable correspondence between the two tech- 
niques. The unreliability in both scores undoubtedly results in a 


TABLE 16 

CoEFnCIENTS OF CORRELATION BETWEEN THE OBJECTIVE AND ESSAT 
ioRiia OP THE Test of Science Reabonino and Undebstandino 
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telUometWng onL nLure^Tabl“''l7°‘'’?u-''^’'*“''* 

ItrafsoSr^' “n^rLtnirin 

because o£ the inteCrr“^.‘*^= 

to beliereUm thTr«uirtS"ther““'‘'' 

ability or of intpii;.T« ^ simply a test of reading 

such a result from theTppoaTh uTe d 'xh " K 
examination (item 6 in Table 181 biological science final 

\olvin»» an indeoendent att ^ semester examination in- 

covered by the Test nf «: ' measure some of the objectives 
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TABLE 17 


CoEmCIEOTS OF CoHREI^TlOV OF TeST OF SCIENCE BeASONINO 
AND Understandinq WITH Kelated Tests 


Test 

1 CoaBEunof CoernciEvt 1 

No. OF 
Colleges 

No, OF 
Stcdents 

1 Lowest 1 

1 Righest 1 

1 Mean ' 


1 Pre-Tesi 

ACE Psychological Exammation 

.27 1 

.71 I 

j .56 

1 ^ 

1,010 

Critical Analysis in Reading and 






Writing. . . . . . . 

.29 

.58 , 

i .48 

1 ^ 

592 

Critical Thinking, Form A 

.20 1 

.74 1 

.51 

1 ^ 

1,010 


[ Post-Teit 

ACE Psychological Examination 

.87 

.62 

.42 

5 

, 864 

Critical Analysis in Keadmg and 

1 





Writing. . 

.88 

.49 

.42 

S 

602 

Critical 't hinking, Jb'orm A . . 

.50 ' 

.70 

.63 

5 

864 


TABLE 18 

Mescellaneods Correlations of the Test of Science 
Reasonino and Understanding with Other Factors 


FacTOB 

DescaiFTios or Gaou? 

N 

1 CoaaxLATtoM 

For Meo 

For Women 

1. ACE Psychological Ex- 
amination 

Entering freahniea in 
large state university 




Q score 

439 ! 

.36 

.33 

L score . 


4S9 

.60 

.51 

T score 


439 

.53 

.60 

2. CEEB Sdiolastic 

Aptitude Test 

' Entering freshmen in pri- 
1 vate university for men 

352 

.46 


3. CEEB Scholastic 
Aptitude Test 

End-of-year freshmen in 
private university for 
men (essentially same 
group as in 2) 

315 

.60 

1 


4. Cooperative Heading 

Test 

Entering freshmen in pri- 
vate tmiversity for men 

341 

.67 


6. Nelson-Uenny Beading 
Test 

Entcrbg freshmen in pri- 
vate college for women 

137 


.35 

6. Biological science final 1 
exammatioD 

End-of-year freshmen in 
private college for 
women 

89 


.72 

7. General education grades 

Juniors and seniors in 
lar^e state university 
(science majors) 

42 

.35 

(men and women) 

8. h»on-general education 
grades 1 

Ditto 

' 42 

1 

.50 

(men and women) 
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science approximates one-fifth of the general education work 
taken. These data are consistent with expectations. 

Gains over the Freshman Year 

As was generally true with all tests, the use of the Test of Science 
Reasoning and Understanding in pre-testing was far more exten- 
sive than in post-testing. Good intentions were all too frequently 
defeated by the difficulties involved in scheduling. Form A was 
extensively used over a one-year period with freshmen, and the 
gams made over the penod are summarized in Table 19. This form 


TABLE 19 

Mean Gains op Science Rbabovino and UNDEESTANDiNa, 

ijomr A. CLASsmEo Acco&dinq to Pbe-Test Standinq 


L-fTTUU-T I I-nruiAT iMmitLY 

towGEOTO Mrooix 

I I GlOUP G%OVT Cadct TotALGlOCT 







these gain7 like thos^f several initial ventures indicated that 
negatifeWwith °*er study tests, were correlated 

tests used precluded\d'^"** Variation in the academic aptitude 

scores was done by covarianre an^l P"^*^ 

to some question because of th ^ procedure was open 

were less understandahl#. ^ ”3ture of the data, and the results 
of Table 19. ° teachers than the descriptive approach 

In order to arrive at this tahu 

tributing forty-five or more ore- “lieges con- 
students were thrown ^ • i scores for the same 

divided as nearly as possible into r was then 

for each college were then sorieH • groups. The students 

g then sorted into these groups. The nature of 














129 


OBJECTIVES IN SCIENCE 

the distribution of the students among these groups for a given 
college reflects the ability level of the students entering that col- 
lege. At the same time it is possible to compare any particular 
group in one institution with the corresponding group in another, 
since the pre-test means are essentially the same. The general pat- 
tern of decreasing gain with higher pre-test scores is apparent. The 
total group gains for most of the colleges represented are statisti- 
cally significant, although they are not large numerically. 

Institutions 5 and 10 have particularly interesting results. Col- 
lege 5 is a liberal arts college with a rather highly selected student 
body, while College 12 is a large state institution. Both show rather 
large gains for the low, low-middle, and middle groups. The more 
selective institution is no more successful tvith its very able stu- 
dents (gain of .89) than the state institution (gain of l.Sl). College 
13, heavily weighted with initially low students, shows only moder- 
ate gains (4.19) for this low group and the more able students 
actually lose ground. The variation in the gains suggests that 
general education programs do differ in the extent to which they 
promote change in the abilities measured by this test. Whether 
those differences are due to differences in emphasis, to quality of 
instruction, to the nature of the science courses, other factors, or a 
complex of many of them, it is impossible to say. It did appear 
that those institutions wherein the larger gains were registered 
were ones in which the class assignments and the laboratory pro- 
vided experience more obviously related to the objectives than was 
true in the other colleges. Reading and analysis of science ma- 
terials, exercises specially prepared with something very like the 
pattern of abilities of the Test of Science Reasoning and Under- 
standing, and tests requiring these abilities, were more apt to be 
a part of the regular instructional program of those colleges mak- 
ing the larger gains. 

It is significant that in the institution in which the largest gain 
^vas recorded, the Test of Science Reasoning and Understanding 
was regarded as so closely related to the objectives of the course 
that it was used as one-half the final examination. Furthermore, 
there was no comment from the students, who could not distin- 
guish between this and other test materials developed specifically 
for the course. 
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In Table 20 are sho^m mean gains for subgroups determined 
by both ability (ACE Psychological Examination) and pre-test 
score on the Test of Science Reasoning and Understanding. It is 
evident from this, once again, that students low on pre-test make 
the largest gains, and it is also evident that the combination of low 
pre-test with high ability yields the largest gains. More significant, 
perhaps, is the fact that, except for one classification of high ability 
and high pre-test, the gains for students whose ability ranking 
equals or exceeds the pre-test status, are rather sizable. Small gains 
are argely characteristic of those whose pre-test rankings are 
higher than the ability level rankings. 


TABLE 20 

Science RE^aoviNO and UNDESSTANDiNa, 
PsTC^OMMCAl AcCORDINO TO QdARTILE RaNK ON IRE ACE 

CHOLOCICAI E^hnation tre Test or Science Reasoning and 
Understandino. Form A. Pbe-Test 


_ ACE 

«TCHOU)OCCAL 
ExAumAiroN 
Qvabtilx Ri'tx 


First (low) 
Second 
Third 
Fourth 

Total grouD 




11 67 

12 00 


5 29 
5 38 
7 02 


4.75 
2 56 


Gam 


- 1.00 
2 C7 
-1 00 
S.8I 

2.26 


Gam 


5.9S 
5.08 
6 85 
5.46 

5.84 


Taken 

intoVoup°s Lord,™'' the freshmen were divided 

during the £reshman"year Th ‘^^"ration courses carried 

Test of Science ■' ft^hmen who took Form C of the 

January 1953 were separfted"!^"^^™'^'"^ T 
what general education courses the'' '>cP™dn'g upon 

diSenfc 

in average ability of the groups aTtaf"" random variations 
covariance method was mad^ ’’’’ 

CEEB Scholastic Aptitude Lr« Nat i 

p ue scores. Natural Science pre-test and 
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TABLE 21 

Gaiks ok Test op Science REAsoKiNa and Unoehstandinq (Form C) 
IN Five Core Courses in a Freshman Class 


Gbottp 

Fboqbau 

N 

Mean Scores 

Gain 

I»rpBOT*« 

UE'fT 

Index 

Pre Test 

Post Test 

1 

Physical science only 

17 

26 2 

31 S 

S 8 

+ 28 

* 

Biological science only 

9 

26 9 

28 4 

1 5 

-1 27 

s 

Social science only 

99 

27 5 

80 4 

2 9 

- 23 

4 

1 and 8 

78 

25 8 

31 0 

5 2 

+ 37 

B 

2 and 3 

86 

24 8 

32 4 

7 6 

+ 03 


post test scores, a predicted mean for each group was calculated 
The results indicate that the differences in the groups would occur 
by chance more often than once in twenty times, perhaps once in 
seven or eight Hence there is no strong evidence that success in 
Natural Science, Form C, is significantly affected by any of these 
course combinations, although physical science alone or in com 
bination with social science did produce somewhat better results 
Study No 2 —At one university, comparisons were made for 
groups taking various science courses and no science at all Ad 
justed mean scores were determined and used in the compansons 
These results, shown in Table 22, indicate that students taking 
either of the available general education science courses attained 
gains over a year which are greater than those for students taking 
no science or a science course for prospective teachers 
Study No 3 —In another college, students with both pre and 
post test scores were divided into groups on the basis of the amount 
of science taken Those with no science showed a mean gam of 1 
point over the year, those with 9 to I^credits in biological science 


TABLE 22 

Comparative Analtses of Aciuevement on the Test or Science 
Eeasonino and Understanding* 


Students EoroOed m 

Mesn 

Pre-Test Score 

Adjusted Afesn 

Post Test Score 

Biological science 1 and 2 

22 42 

25 88t 

Physical science 1 and 2 

27 47 

26 12t 


20 S3 

24 78 

No science 

23 78 

24 13 

hlean for all students 

23 77 

- 


• Uomld G WillAce “A Report of the Drtke E»*lu*t»<ia Study la CeneteJ EduMbou (Mimeocrtphed. 
Ureke Uttivtrfily De«MoDet,^ws lOM) 
t Sigtu£c*nt ct J)l fereL 
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and 5 credits in physical science had a mean gain of 4.5. With one 
exception, there was a gradual increase in mean gain with the 
amount of science taken. 

Study No. 4.— From Brooklyn College, Heil reported that the 
Test of Science Reasoning and Understanding was used as one of 
several instruments for comparing students in an experimental 
general education program with those in the regular program. For 
students in the two programs matched on entrance qualifications, 
high school attended, sex, and major field, results at the end of the 
sophomore year were favorable to the experimental program al- 
though the differences were not statistically significant. 

latger group gains reported was for 
199 students enrolled in a general education science course for 
one semester. The emphasis in instruction was related closely to 
the objectives involved in the test. The mean gain on the Physical 
cience orm o the Test of Science Reasoning and Understanding 

°£ the eight 

arQii<(.d'l^°tI.^ ranging from 5.2 to 9.6. Considerable interest is 
class of 2R sf a largest gain. 9.6, was obtained with a 

man class hi s I 'r'’ '°P °£ the fresh- 

nre-test Scholastic Aptitude Test. The mean 

classes This •°th ' about 6 score points above the other 

esrSn^U ! , h gt°“p made the larg- 

by fn instrurtn° section taught 

were pre- 

A mean gain of 5.1 was found 

those taking simultaneously a r “■ “ ‘he fact that 

strated no greater gain thanlh “ practical logic demon- 

ing suggests that the transfer ™lue"of ^c ' 
very great. c ot a course in logic may not be 

Study No. 7 .—At one college the Tesr nf c • ti - j 
Understanding in Physical n- i • I Reasoning and 

both the biological and physical sr” was used with 

period. Both ^oups madeStom 

gain made in each course m L test -1 

test related to that area was 
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significantly higher than that m the other course The indication 
IS dearly that achievement of the objectives of science reasoning 
and understanding involve both knowledge of or familiarity with 
the material and ability to reason in it 

T est Performance Related to Number of Years in College 
Study No S— Table 23 shows for three colleges the retesting 
results at one and two year intervals The same students were 
retested at the two-year interval and the one year results are based 
on a subgroup of these same students The pattern here, found in 
TABLE as 


One- and Two Yeah Test Resoets for Science Reasontno 
AND Undesstanmno Test Form A 


CoUEOB 

I Pat-Tts* 1 

[ On£ Yua Latxb 

1 TVo Yubs Laicb 

N 


SJ> 

N 

Uean 

SD 

N 1 

Mean 

SD 

A 

79 

23 72 

5 34 

55 

28 78 1 

5 47 1 

79 

29 95 

5 48 

B 

64 

22 75 

5 77 

smtcbed forms I 

64 

28 28 

5 32 

C 

66 

24 47 

4 59 

1 66 

j 28 77 

I4 86] 

66 

29 86 

6 19 


Other cases also, is a gam o£ 4 to 5 raw score points the first year, 
followed by a smaller gam the second year Partly this reflects the 
tendency to take general education science courses in the freshman 
year, but it was still true to a lesser degree where science was con 
tmued for two years, thus bearing out the tendency for higher 
pre test scores to be associated with smaller gams 
Study No 9 —At another college, freshman and senior women 
were matched in terras of ability on the ACE Psychological 
Examination and the performance on the Test of Science Reason 
ing and Understanding These data are shown m Table 24 
The seniors had taken a minimum of two general education 
science courses dunng the preceding four years The difference m 


TABLE a 

Performance of Matcdeo Groups on the Test of 
Science Reasonino and Unoebstandino Form A 


Group Trited 

Sfean 

SD 

101 enlermc freshmen* 

101 seniorsf 

24 98 

4 50 

SI 00 

6 Si 


• ^pUmber IMl 

tTeiUd Fcwuvf 1951. 
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means is approximately 6 points and indicates a real improvement. 
A similar investigation tvith another matched group yielded a dif- 
ference in means of 7 points. 

Results of Experimental Studies 

Tables 25, 26, and 27 show the results of studies comparing 
science classes wherein the objectives involved in the Test of 
Science Reasoning and Understanding were emphasized with 
dasses in which no overt attention was given to these objectives. 

ese investigations were arranged by committee members to see 
whether their newly developed ideas about instruction would 
result in any greater change in students. 

In most of these comparisons the gain for the general education 
course in which the instructor gave special attention to the objec- 
jves was larger than for the non general education courses or for 
* Pi™arily content emphasis, 
mean rath'" 1 ^^'* “dogy shows an adjusted post test 

Tn Tabll ^7 biology course. 

27 a physical science course which supposedly gave no 

TABLE S5 


Gboct Tested 

N 

Pbb- 

Test 

SD 

Pos 

••TEaT 

Adjcsted 

Post-Test 

physical science course* 
Biological science coursef 

65 

S2 

23 1 

24 9 

5 9 

7 9 

SO 4 

23 6 

6 1 

7 4 

29 5 

25 1 

A genetil education couik with i ! 

t A general education comae «niphaau on tlw aoentifie method 

TABLE 26 

80NINO 

HGE 


Gsottp Tested 

N 

Pbe- 

Mean 

Test 

SD 

Post 

Test 

Ao/usted 
Post Test 

Biology* 

Botany 

Zoology 

Chemistry 

203 

83 

103 

287 

29 0 

24 5 

29 3 

24 4 

7 4 

6 4 

5 3 

6 6 

31 1 

25 5 

80 7 

25 2 

7 0 

6 9 

5 8 

29 5 

26 8 

28 9 


A KeoerU eduction course enth emphaei, oi 
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TABLE 27 


One-Yeaii Gains on the Test of Science Reasoninq 
AND Understanding, Form A, in a Junior College 


Gttocp Tested ^ 

! 

N ^ 

[ Pbe*Test 

I Post Test 

Adjusted 
Post Test 
Me.\n 

Kfeui 

SD 

Kfean 

( SD 

Biological Science lOI* 

86 

22 6 

6 4 

27 I 

1 6.6 

1 29 7 

Biological Science 111* 

72 

25 S 

7.2 

2S 6 

! 7 1 

1 29 0 

Physical Science 101 f 

S7 

26 1 

6 1 

' 28 6 

' 6 3 

29 0 

Chemistry 201 

SO 

25 5 

6 1 

j 28 8 

6.1 

28 7 

Physical Science 101 f 

31 

25 2 

6 1 

' 28 6 

6 4 

27 8 

Chemistry 201 

SO 

26 7 

8 8 

, SO 5 

6 9 

27 5 

Biological Science lllf 

32 

27 1 

5 7 

28 1 

7 2 

26 9 

Biological Science lOlf 

64 

I 26 6 

7.7 

, 28 2 

7 0 

26 4 


* A general education course witb emphasis on scientific method, 
t A general education course 


unusual attention to objectives of the Test of Science Reasoning 
and Understanding and a chemistry course produced gains com- 
paring rather favorably with courses attempting special emphasis 
on the objectives of the Test of Science Reasoning and Under- 
standing, 

The techniques used by the teachers of the experimental sec- 
tions reported on in these tables varied considerably. The teacher 
of the physical science course in Table 25 regularly used current 
science articles as part of the class work. These articles were read, 
analyzed in small groups, and issues resolved by experiment or by 
use of reference material Extensive use was also made of exercises 
in which attention was focused on the abilities defined in the 
Guide for Construction of Evaluation Items (see pages 108-9). 

In the course in biology in Table 26 the abilities in the guide 
were discussed with students; also illustrative exercises and reading 
passages were used to develop an awareness and understanding of 
them. In recitation and discussion students were continually 
pushed to relate their thinking to the problem recognition, infor- 
mation seeking, hypothesis formation, conclusion checking, and 
attitude awareness developed in the guide. 

The extent of attention to the scientific method in other experi- 
mental work was not reported in detail, and it apparently varied 
considerably from instructor to instructor. The lack of control of 
the instructor variable in all of tliese comparisons is quite obvi- 
ously a major weakness, but one over which the central office of 
the Study had no control. 
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means is approximately 6 points and indicates a real improvement 
A similar investigation with another matched group yielded a dif 
ference in means of 7 points 

Results of Experimental Studies 

Tables 25 26 and 27 show the results of studies comparing 
science classes wherein the objectives involved in the Test of 
Science Reasoning and Understanding were emphasized ivnth 
classes in which no overt attention was given to these objectives 
These investigations were arranged by committee members to see 
whether their newly developed ideas about instruction would 
result in any greater change in students 
In most of these comparisons the gam tor the general education 
course in which the instructor gave special attention to the objcc 
trees was arger than for the non general education courses or tor 
e general education courses with a primarily content emphasis 
^ shows an adjusted post test 

^ general education biology course 

e a p ysical science course which supposedly gave no 

TABLE is 


Gmup Tested 

N 

{ P«b-Twt 

I Post Test 

Adjdsted 

Post-Tesi 

Mesn 

SJ) 

Tlf..., 


^iiys cal 8c ence course* 

B ological sc ence coursej 

65 

S2 

28 1 

24 9 

5 9 

7 9 

SO 4 

23 6 

6 1 

7 4 

29 5 

25 1 


SONINO 

HOE 


Gbocf Tested ' 

N 

Pu 

Tbt 

SD 

Post 

Test 

Ad tJSTED 
' Post Test 

11 ology* 1 

Botany 1 

Zoolo^ 1 

Chemistry I 

203 

83 

103 

287 

! 29 0 

24 5 

29 d 

1 24 4 

7 4 

6 4 

5 S 

6 6 

31 1 

25 5 

30 7 

25 2 

7 0 

6 9 

5 8 

29 5 

26 8 

28 9 

26 5 
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table 

O-k-Yeui Gmn3 0-j the Test 

AND Understanding Form A, in a _ 


Gbottp Tisted 

Biolog cal Science 101* 
Biological Science 111* 
Physical Science 101^ 
Chemistry 201 
Physical Science 101^ 
Chemistry 201 
B ological Science lilt 
Biological Science lOlf 


Pbc-Test 
tean 1 SD 


Post-Test 


* A general educat on courae with etnpliaa a on sc cntific mcth 

t A general education course T>«/»er«nin?T 

f thf^ Test of Science Reasoning 
unusual attention to objectives ot tne * ^^oduced gams com 
and Understanding and a chemistry course jpecial emphasis 
paring rather favorably with courses and Under 

on the objecuves of the Test of Science Reason g 
standing , experimental sec 

The techniques used by the teac Qj, 5 ,(ierably The teacher 
tions reported on in these tables ^ 25 regularly used current 
of the physical science course in Ta -j.j^g°e articles were read, 
science articles as part of the class experiment or by 

analyzed in small groups, and issues r^^ made of exercises 

use of reference material ^ abilities defined in the 

m which attention was Items (see pages 108-9) 

Guide for Construction of Ev abilities m the guide 

In the course m biology m ., „tive exercises and reading 
uere discussed with “ awareness and understanding of 

passages were used to develop ^ students were continually 

them In recitation the problem recognition, infor- 

pushed to relate jonnaiion, conclusion checking, and 

mition seeking hypotnesis _,,,i_ 

aituudc ^tm°'’to the scientific method m other expcri 

Tlic extent of j detail, and it apparently laricd 

"emS to instructor The lack of control of 

dm nsmicmr sanablc in all of these comparisons is quite obi. 
oiiVy a major weakness, but one over which the central oITice of 
the Stud) had no control 
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In an entirely independent study, Perlman investigated the rela 
tive effectiveness of three different methods of teaching scientific 
thinking His results, like some reported here, were rather incon 
elusive, although apparent differences seemed to favor the use of 
contemporary problem matenal in the laboratory More to the 
point here, however, are some of Perlman’s summary implications 
After remarking that his results provide “Further indications here 
that scientific thinking and scientific approach to problems can be 
learned through direct classroom planning and procedures for the 
purpose, he points out that there are innumerable variations of 
the instructional approaches used He further recommends the 
need for studies in which not only the laboratory periods, but all 
class activities emphasize problem solving processes ’’ He also raises 
the question, ‘ To what extent can a single course or section of a 
course with emphasis on scientific thinking and procedures among 
other things, overcome habits of many years of past school expen 
ence of students based upon processes that involve essentially 
memorization and recall? * These remarks of Perlman are much 
to the point and emphasize issues raised by various expenmental 
studies in this project 


Conclusions and Implications 
From the rather heterogeneous studies and data, even though 
only partially reported here, it would be rash to draw more than 
entative conclusions Such data as bear on the point confirm the 
committee judgment that the Test of Science Reasoning and 
rwT'fn ^'^dmg skill or intelligence 

th fh science courses made greater gams 

an those who did not, and those who took more science tended 

whS'" who took but one course In those cases 

where a valid comparison could be made between a general educa 
t on science course and other scence courses, the advantage lay 

~ *<= dtfferences were 

of Snec 1 ** suggested that students svith a combination 

of specialized and general education science made larger gams than 

Colltge Physical' 

E^pmmental Edficalion, March iSs pp ffitis, ^ PducauOD Th, Journal of 
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those taking either type exclusively. More striking, perhaps, is the 
marked variation in the gains found in different colleges -Nvith 
markedly different programs. In some colleges, even after allow- 
ance is made for the level of student ability, the gains made are 
relatively small compared with other colleges. It also seems reason- 
able to conclude that most science programs could achieve more in 
regard to these abilities than is currently the case. Observations in 
visiting science classes on some of the campuses and in talking with 
teachers, buttressed by the judgment of the Science Committee 
members, suggest that in most programs little more than lip service 
is paid to the objectives. Even in the classes where definite attempts 
were made to teach for scientific reasoning and understanding, the 
new instructional practices must be regarded as exploratory and 
sometimes rather superficial. It is quite conceivable, therefore, 
that the largest gain recorded in our findings is not indicative of 
the possible gain. 

Mean gains of 5 to 10 points on the test may seem rather trivial 
and be taken either as an indication that the test is unsatisfactory or 
that little has been or can be done about such objectives. In re- 
buttal it should be noted that 5 to 10 points is 10 to 20 percent of 
the maximum test score of 50 points, which is a respectable gain 
even for broadly conceived achievement tests. Only in highly 
specific factual examinations based on materials actually covered 
are greater gains recorded, and usually there is a marked loss in 
such specific knowledge within a few months after completion of 
the course unless study is continued in that same area. The gains 
on such objectives as those studied here must be much more nearly 
those recorded in similar periods for improvement in reading and 
writing. Such abilities develop slowly under even the best of in- 
struction. 

The results of the testing and research side of the science 
venture are summarized in the folloiving points: (I) Four forms of 
a promising new instrument for measuring certain general educa- 
tion outcomes have been made available; (2) such data as have 
been collected by use of this instrument in a number of general 
education programs indicate that some progress is made by stu- 
dents in regard to the abilities measured; (3) there is considerable 
variation in the kind of results obtained in the various programs 



138 


GENERAL EDUCATION 


and there is some indication that the amount of success is due to 
the attention given to the objectives 
The implications of the project on the instructional side may 
have more long range value than the activity supposedly repre 
senting the major concern of the committee In the handbook, 
Science Reasoning and Understanding, there are numerous indica 
tions of ways in which the three objectives studied by the com 
mittee can be made to be vital parts of the daily classroom experi 
ence of each student The development of essay materials, partly 
for validation purposes and partly because of a growing concern 
that some teachers are all too likely to use objective techniques to 
the exclusion of the essay approach, has much merit The con 
sistent usage of structured essay exercises, as outlined in the hand 
book for class work and discussion, for homework assignments, or 
for testing would give students more opportunity to develop the 
three abilities than any amount of textbook reading or instructor 
lecturing Dr Donald Decker, chairman of the Division of Sciences 
at the Colorado State College of Education, undertook to develop 
in a graduate seminar composed of classroom teachers of science 
«say ideas and materials similar to those developed in the Science 
Committee He reported a great deal of enthusiasm on the part of 
^aph^^^^”’ summarized his reactions in the following para 


stu^ slirh°^ ^ opinion that the amount of time and effort spent in a 
been ,1" American Council on Education has 

in the studv n p t. ^ value as those who have participated 

mht ‘hat Irk to help 

« ‘hese methods 

the studv nf Pirtsiiiof f "*^d a great deal more significance m 
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•Quoted from a pe,»„al loner u, the S.„d, Dneetor August 15 1S51 
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Intercollege Committee on Humanities Objectives Peter Coffin, 
Wnght Junior College, Robert F Davidson, University o£ 
Florida, Helen E Elcock, Kansas State College of Agriculture 
and Applied Science, James Fisher, Boston University, John 
Kendall, Muskingum College, V E Leichty, Michigan State 
College, Robert Miller, Florida State University, James Nardin, 
Colorado State College of Education, John Pletz, Wnght Junior 
College, James Rice, Stephens College Paul A Rockford, Col 
gate University, Paula Thibault, Educational Testing Service 


I T IS AGAINST a backdrop of confusion in regard to aims, methods, 
and content of humanities courses that the work of the 
Humanities Committee of the Co operative Study of Evalua 
tion in General Education must be examined The teachers from 
participating colleges, who met regularly in the committee over a 
period of three years, represented almost every possible type of 
course organization in the area of the humanities That these 
people could reconcile differences and maintain a lively interest in 
a prolonged project is evidence of a desire to find some unifying 
principle against which courses might be compared It is further 
proof of a growing general concern about evaluation practices 
found on college campuses in respect to the humanities The mag 
nitude of the differences which at first divided the committee can 
best be illustrated by giving some attention to tlie way inimanitics 
courses have developed in American higher education 

Historically the content of the humanities consisted of the more 
human studies emphasized during the Renaissance in contrast to 
the thcocentric curricula of the Middle Ages Howeier, in more 
recent interpretations, tlie humanities include any or all of litera 
1S9 
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ture, philosophy, music, architecture, drama, ballet, painting, and 
quite frequently religion and history. The precise combination of 
content, the manner in which it is treated, and the objectives 
sought differ widely from institution to institution, and apparently 
depend mainly on the interest and training of individuals to whom 
responsibility for the development of a course is assigned. 

Perhaps the most frequently observed college course in the 
humanities is some variant of a survey of some one or several 


aspects of culture— in its earliest form a survey of the major literary 
developments of America or England. Of somewhat more recent 
vintage is the History of Civilization course (most generally de- 
fined with the adjective Western) in which students studied about 
the men and events which have shaped the Western European 
mind. Although still somewhat in vogue, these broad survey 
courses have quite frequently mutated into studies in greater depth 
of particular cultural epochs such as Greece in the Age of Pericles, 
the Middle Ages (usually the thirteenth century), the Renaissance 
and Reformation, the Age of Enlightenment, and modern times. 

Combining elements of the survey of literature and the cultural 
epoch approach, some recent humanities courses have attempted 
to present each of a few major periods through a consideration of 
the major artistic and literary developments. Thus the sculpture, 
arc itecture, philosophy, and drama of Hellenic Greece are shown 
as interacting manifestations of the same spirit in contrast to 
sirnilar phenomena of the Middle Ages or the twentieth century, 
epreseniing a completely different approach to the humanities 
t lose courses which use content as only a means to develop 
certain intellectual skills. Courses based upon Great Books, for 
mp e, are more designed to train students in a specific dialecti* 
° provide information. Other courses 

o£ a fcw Selected documents in music or 
with 'mv . rf”'" ‘•'e grammar of those subjects, not 

tmhiiM is ''•"r intellectual 

mr^rnm. r tnfrequently such courses are arranged his- 

toncall) while emphasirmg the disciplinary aspects. 

snchXn h r ™ the round that 

f “Phoeeote years tends to repel stu- 

dents from further contact w.th the humanities. They hold that a 



OBJECTIVES IN THE HUMANITIES 141 

primary objective of humanities courses is to develop an apprecia- 
tion of literature and the fine arts which will enrich the individ- 
ual’s total adult life. Students should be taught to like or ap- 
preciate such things before becoming critical and analytical. One 
course, organized but recently, merely introduces students to a 
variety of humanities experiences pretty much at a feeling level. 

Other teachers have insisted that the essence of humanistic study 
is to be found only in creative work. They argue that an hour 
spent in the studio working with paints would be worth many 
hours studying about works of art or even studying works them- 
selves. They contend that most education is too passive—that the 
lasting values are derived from active participation. Thus, they 
would provide studio courses of various sorts. 

One can also detect other fundamental issues which serve to 
divide humanities courses and teachers. From one point of view 
humanities are regarded as means by which values are taught to 
college students. Proponents of this concept theorize that students 
are taught to be coldly rational in the solution of their problems 
in the sciences and social sciences. In the humanities they are 
shown that all ultimate solutions of the rational sciences rest upon 
value assumptions, and they are encouraged to examine some of 
those values. In place of relying upon the methods of science, the 
humanities utilize the methods of religion, philosophy, and esthe- 
tic insight. 

Other divisiveness is found in respect to the kind of documents 
studied. Some teachers believe that students will probably never, 
in adult life, make much use of primary historical documents or 
certain esoteric paintings or musical selections. To compensate for 
this anticipated lack, but with the hope that students’ level of taste 
will improve, unusual, difficult, and abstract documents are taught. 
Other teachers feel that a general course should provide experience 
with the materials students will be most likely to encounter as 
adults. To them, a study of television programs or motion pictures 
would be more appropriate than a study of Gregorian cliants and 
Shakespeare’s sonnets. 

Such disparate conceptions of the nature of the humanities is re- 
fiected in the teaching techniques and evaluation practices. The 
sur\’cy-typc course organization has placed a premium upon Icctur- 
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ing and testing for knowledge of facts. More analytical courses 
ostensibly rely to a greater extent on discussion, and conceive of 
evaluation as determining whether students can creatively organize 
a paper in which a specific document is criticized. The fine arts 
appreciation courses have perhaps the most difficult time with re- 
gard to teaching or testing. Since teachers do not want to dictate 
taste, they must expose students to a variety of selections without 
giving them much interpretive comment. Testing must be re- 
stricted to appraising knowledge of fact instead of the principal 
course objective of appreciation. 


Common Objectives of Humanities Courses 
The interest and concern of the Humanities Committee in 
reconciling some of these differences and in finding appropriate 
evaluation techniques were directed to finding common aspects of 
humanities courses toward which an evaluation study could be ori- 
ented. The area selected was stated as an objective in which all 
courses should be interested: "To understand and enjoy literature, 
art, music and other cultural activities as expressions of personal 
an social experience, and to participate to some extent in some 
orm 0 creative activity.”^ This objective seemed to represent the 
major common goals of survey courses, critical courses, appreciation 
courses, and studio offerings. The objective also appeared to be test- 
a e without demanding a common fund of knowledge on the part 

-iiirl an essential quality in a cooperative enterprise with 

sudi diverse points of view represented. 

elements of the objective seemed to consist of 
dpntc frp* participation. There was a feeling that stu- 

of some ^ values of humanities courses because 

mierLv ^t«dy. Such dislikes 

wmL of iL r premature analytical study of 

cuUurafno^nn m''; instrumental pmctice, or the 

"nullinff the! I P^^*^^°n^rs of the humanities were not quite 
and S red tT l' predilections e^dsted 

be of conre . achievement of course objectives, they should 
be of concern to teachers. Appraisal of attitudes toward the 
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humanities might provide some indication of the degree to which 
students could accept the values of the humanities. 

Participation in the humanities should indicate something 
about an individual’s feeling toward art, music, literature, and the 
like on the ground that a choice between two possible activities 
would probably serve as an index of attitude or feeling. Increasing 
the quality of participation in these activities should be a legiti- 
mate concern of humanities teachers in general education courses. 
If some appropriate index could be obtained of the amount, kind, 
and degree of student participation in the arts, teachers could 
know to what extent they were affecting their students’ behavior, 
and could adjust courses to better achieve this goal. 

Humanities teachers, however, could not be concerned ex- 
clusively with changing students’ likes or dislikes. They should 
also affect students’ skill in understanding the arts as well as in- 
creasing their knowledge about them. Techniques were needed to 
measure students’ ability to deal, at an intellectual level, with 
various documents in the humanities and to assess whatever com- 
mon body of knowledge might be taught in the cooperating 
schools. The remainder of this discussion will be concerned with 
the way the Humanities Committee sought to evaluate student 
progress -with respect to those aspects of participation, attitudes, 
analysis, and knowledge, which were of common interest to all its 
members. 


The Humanities Participation Inventory 
There are many ways by which student participation in humani- 
ties activities might be assessed. Attendance checks at concerts, art 
displays, or lectures would be one method. A study of library call 
slips would be another, as also would be a survey of use made of 
listening rooms or studio facilities. Valuable as such means are, 
they have the disadvantage of being laborious to employ. A study 
of library call slips at one large institution, for example, required 
the full time of a research clerk for two entire weeks to enable 
teachers to obtain quickly the information with which to study 
student participation. An easy technique to accumulate data was 
needed. 

One possibility involved the preparation of a rather complete list 
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art exhibits or looking at prints; drawing, painting, or doing 
handicraft; reading philosophical or religious essays or books; and 
discussing such problems. Students were asked to indicate how fre- 
quently they participated, the degree of enjoyment they derived 
from the activity, and the seriousness with which they participated. 
A scoring device ^vas prepared so that students could respond to 
the items on a standard live-choice IBM answer sheet, which per- 
mitted quick scoring and detailed analysis. 

The items as printed in the final form of the Humanities Partici- 
pation Inventory took the form: 

You read short stories or novels 

1. 1. never. (If you choose this answer do not mark anything 

for items 2 or S.) 

2. occasionally. 

3. frequently. 

2. 1. with little or no enjoyment. 

2. with a fair amount of enJoymenL 

3. with great pleasure. 

3. 1. just for the story. 

2. paying some attention to plot and characterization. 

S. making a detailed examination of the idea and structure 
of the work in its theatrical and literary contexts. 

Attitudes toward the Humanities 

The second affective concern of the committee was with stu- 
dents’ attitudes toivard the humanities. There ivas some reason to 
believe that many students had unfavorable feelings about the 
humanities. The task, then, was to devise means by which these 
feelings or attitudes might be assessed, with a view to providing 
information for course planning on the basis of ^vhatever senti- 
ments were discovered. It should be pointed out that this was the 
least successful of all the committee’s ventures. 

In attempting to do this, a great number of issues had to be re- 
solved some of which eventually proved beyond the resources 
available to the committee; 

1. The first issue had to do with the nature of attitudes. The 
term attitude is used in such a variety of ways as to preclude easy 
general definition. Meanings range from fundamental life princi- 
ples to vague feelings about certain objects. For its purpose the 



144 


GENERAL EDUCATION 


of humanities activities to be submitted to students ivith the re- 
quest that they indicate how frequently they engaged in each 
activity Such a list (which would run into many pages), however, 
might well defeat its purpose either by boring students or by dis 
couraging them by showing the vast areas in which they had had 
no experience A modification of this idea— consisting of a list of 
actiMties to i\hich students were to respond three separate times 
(1) in terms of the availability of each activity or experience, (2) 
how frequently they engaged in each activity, and (3) the intensity 
with which they participated— was developed and tried out with 
students The lists of items, while not exhaustive, nonetheless con 
tamed too much to maintain student interest Motivation lagged 
particularly the third time the students viewed the same list of 
activities 


Experiences w ith this inventory revealed that assessing participa 
*neans as easy a task as had at first been anticipated 
n a ition to tiring of items seen three successive times, students 
oun u impossible to respond to ceruin items because they were 
so r om eir own cxpenenccs or because the alternative re 
j unrealistic For example, one of the senes of responses 
SK (1) not at all, (2) twice, (3) 

times, (5) more than twelve times, dunng the 

chori'^ ^ r *<= >'"'"■ "D-tected or prepared 
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art exhibits or looking at prints; drawing, painting, or doing 
handicraft; reading philosophical or religious essays or books; and 
discussing such problems. Students were asked to indicate how fre- 
quently they participated, the degree of enjoyment they derived 
from the activity, and the seriousness with which they participated. 
A scoring device was prepared so that students could respond to 
the items on a standard five-choice IBM answer sheet, which per- 
mitted quick scoring and detailed analysis. 

The items as printed in the final form of the Humanities Partici- 
pation Inventory took the form: 

You read short stories or novels 

1. 1. never. (If you choose this answer do not mark anything 

for items 2 or 3.) 

2. occasionally. 

3. frequently. 

2. 1. with little or no enjoyment. 

2. with a fair amount of enjoyment 

3. with great pleasure. 

3. I. just for the story. 

2. paying some attention to plot and characterization. 

3. making a detailed examination of the idea and structure 
of the work in its theatrical and literary contexts. 

Attitudes toward the Humanities 

The second affective concern of the committee was with stu- 
dents’ attitudes toward the humanities. There was some reason to 
believe that many students had unfavorable feelings about the 
humanities. The task, then, was to devise means by which these 
feelings or attitudes might be assessed, with a view to providing 
information for course planning on the basis of whatever senti- 
ments were discovered. It should be pointed out that this was the 
least successful of all the committee’s ventures. 

In attempting to do this, a great number of issues had to be re- 
solved some of which eventually proved beyond the resources 
available to the committee: 

1. The first issue had to do with the nature of attitudes. The 
term attitude is used in such a variety of ways as to preclude easy 
general definition. Meanings range from fundamental life princi- 
ples to vague feelings about certain objects. For its purpose the 
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of humanities activities to be submitted to students with the re- 
quest that they indicate how frequently they engaged in each 
activity. Such a list (which would run into many pages), however, 
might well defeat its purpose either by boring students or by dis- 
couraging them by showing the vast areas in which they had had 
no experience. A modification of this idea— consisting of a list of 
activities to which students were to respond three separate times: 
(1) in terms of the availability of each activity or experience, (2) 
how frequently they engaged in each activity, and (3) the intensity 
with which they participated— was developed and tried out with 
students. The lists of items, while not exhaustive, nonetheless con- 
tained too much to maintain student interest. Motivation lagged 
particularly the third time the students viewed the same list of 
activities. 


_ Experiences with this inventory revealed that assessing participa- 
tion was by no means as easy a task as had at first been anticipated, 
n a ition to tiring of items seen three successive times, students 
oun It impossible to respond to certain items because they were 
0 tar from their own experiences or because the alternative re- 
nrov’H* Unrealistic, For example, one of the series of responses 
sk (') not at all. (2) twice, (3) 
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art exhibits or looking at prints; drawing, painting, or doing 
handicraft; reading philosophical or religious essays or books; and 
discussing such problems. Students were asked to indicate how fre- 
quently they participated, the degree of enjoyment they derived 
from the activity, and the seriousness with which they participated. 
A scoring device was prepared so that students could respond to 
the items on a standard five-choice IBM answer sheet, which per- 
mitted quick scoring and detailed analysis. 

The items as printed in the final form of the Humanities Partici- 
pation Inventory took the form: 

You read short stories or novels 

1. 1. never. (If you choose this answer do not mark anything 

for items 2 or 3.) 

2. occasionally. 

3. frequently. 

2. 1, with little or no enjoyment. 

2. with a fair amount of enjoyment 

3. with great pleasure. 

8. 1. just for the story. 

2. paying some attention to plot and characterization. 

3. making a detailed examination of the idea and structure 
of the work in its theatrical and literary contexts. 

Attitudes toward the Humanities 

The second affective concern of the committee was with stu- 
dents’ attitudes coward the humanities. There was some reason to 
believe that many students had unfavorable feelings about the 
humanities. The task, then, was to devise means by which these 
feelings or attitudes might be assessed, with a view to providing 
information for course planning on the basis of whatever senti- 
ments were discovered. It should be pointed out that this was the 
least successful of all the committee’s ventures. 

In attempting to do this, a great number of issues had to be re- 
solved some of which eventually proved beyond the resources 
available to the committee: 

1. The first issue had to do with the nature of attitudes. The 
term attitude is used in such a variety of ways as to preclude easy 
general definition. Meanings range from fundamental life princi- 
ples to vague feelings about certain objects. For its purpose the 
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committee accepted a technical definition that an attitude is an 
emotionalized tendency, organized through experience, to act for 
or against something An attitude was thus conceived of as a feeling 
which had to be inferred from overt bevavior It was thought of 
as only a tendency to act— not an act itself 

2 The second issue grew logically out of the first An attitude is 
a tendency to act for or against something What should be the 
nature of the something m which the committee members were 
interested? Was it possible to consider the humanities as a gen 
eralized object composed of many specific attitude objects? Or 
were attitudes toward certain objects so discretely organized in a 
person's experience that they conditioned but slightly, if at all, 
attitudes toward other similar objects? For example, could feelings 
of dislike toward modern artists be combined with feelings of 
distaste toward modem atonal music to yield a generalized attitude 
toward modern art forms which in turn combined with other 
antipathetic feelings about the entire area of the humanities? This 
issue was resolved by assuming that an inventory could be con 
structed which would yield a meaningful total index of a very 
general attitude but which could also be scored to obtain indexes 
of more specific objects The final inventory developed provided 
or a general index and one for each of five major clusters of the 
arts ■' 


3 Another fundamental issue involved the difference between 
a titudes and opinions, but was never successfully met Some means 
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could be considered separately, or should the instrument attempt 
only to measure attitude? One of the reasons that the resulting 
instrument was not successful was that these two conflicting points 
of view were never reconciled. 

4. A fourth issue involved finding an appropriate technique to 
assess attitudes. While the field of attitude testing is relatively new, 
there have been developed a number of techniques, each having 
definite limitations. These might be classified in a variety of ways, 
but perhaps the most important involved the matter of disguise. 
The most clear-cut attitude statements are undisguised questions 
which reveal to the examinee the nature of the attitude being con- 
sidered. The Army, for example, found in World War II that the 
best way of finding soldiers’ attitudes to^vard service in the tropics 
was to ask them whether or not they liked hot weather. However, 
such direct questions may frequently result in dishonest answers 
if a socially acceptable response is clearly indicated. The quickest 
way to find out students’ attitudes toward a teacher would be to 
ask them. However, the implied threat in an unfavorable response 
might prevent students from answering honestly. On the other 
hand, disguised questions may not allow reliable inferences. The 
committee resolved this difficulty by preparing a test of a number 
of items, most of which were undisguised but which, by means of 
a contrived method of response, were functionally disguised.* 
Apparently the solution was not successful, for students by and 
large gave sociaffy acceptable answers even to questions about 
which their feelings were known to be different. 

5. One last issue was in regard to the form items should take. 
Should items be entirely prescriptive, or could some indicative 
statements be used? There was some feeling that safe inferences 
concerning attitudes could only be made from prescriptive state- 
ments of the type, “All students should take courses in the humani- 
ties.” However, the point of view prevailed that valid inferences 
could be drawn from some statements such as, “Modem art is 
rubbish.” 

The committee then proceeded to the development of its in- 
strument. Students were asked to write themes on topics which 

* See chap. 8 on the theory of this technique which was basic to the structure of 
the Inventory of Beliefs. 
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seemed likely to result in expressions of definite attitudinal state 
ments In all, about twenty five hundred such statements were 
extracted from these papers and the most promising refined and 
fitted into a framework based on five major areas of the arts— 
music, literature plastic arts, philosophy, and the combined arts 
Most of the Items included m the final 115 statements of the 
Attitudes Inventory were of a nature that teachers would want 
their students to disagree with them A few, however, were of the 
opposite although these were not to be counted in the students 
score Illustrations of each type of item are included below 
Disagree items (scored) 

I Mast ptKtry seems like a meaningless jumble of svords 

1 The reading of plays is boring 

4 Art^if order or system in the arts is futile 
4 Art IS a frill that society can do without 
R philosophy IS useless 

Most philosophers are intellectual snobs 
Agree Hems (not scored) 

2 The personwbr! helpful in judging a work of art 

mg experience modern painung is missing a reward 
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failed the committee went nnamm 

the validity of concern m .n " record as upholding 

be developed and greater teAniSr techniques should 

future technical resources exploited in the 

Interesting and provocative as the eft 
major interests of the Humanities 

cognitive outcomes of general ediirm members were in 

general education For the obvious reasons 
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of disparity between the content of various courses, nothing could 
be done toward cooperatively testing for acquisition of knowledge. 
However, in two major areas there seemed to be possibility for 
cooperative inquiry. The first of these dealt with vocabulary. 

Humanities Vocabulary Test 

Regardless of whether students study a history of Western civili- 
zation, a series of great books, or a sequence of important pictures 
and musical compositions, they are all expected to communicate 
freely with their peers as they meet them socially and at work. Such 
free communication presupposes a certain common vocabulary, 
regardless of where or how it was acquired. The Humanities Com- 
mittee was interested in finding out the degree to which their 
several courses provided students with a common vocabulary in 
the humanities. Several approaches to such a project were possible: 

1. A few terms might be selected which were common to all 
schools. Several levels of test items would then be prepared for 
each term, ranging from items demanding only simple recall of a 
definition to those demanding insight into a unique situation. 

2. A thorough survey might be made of all terms common to 
courses represented in the committee. From these a representative 
sample might be selected according to the relative emphases ac- 
corded different phases of the humanities. Items testing for these 
concepts and terms might be at almost any or all levels of difficulty. 

3. A third approach might involve coordination of a vocabulary 
test with some other instrument. Thus, if an objective-type test in 
critical thinking in the area of the humanities were developed, an 
instrument testing for knowledge of the terms that test required 
might be a worth-^vhile enterprise. 

Both the first and second approaches were attempted by the 
committee. 

The first instrument constructed attempted to test knowledge of 
a relatively few vocabulary items at several different levels. How- 
ever, the resultant pattern of items gave the impression of inade- 
quate item distribution. The trial form, for example, contained 
clusters of items based upon Machiavellianism, Faust, and 
Oedipus. Twenty items out of sixty dealt with Darwinism, Marx- 
ism, and Freudianism. These topics, while highly interesting and 
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rewarding, were certainly not representative of what was taught 
in humanities courses across the country 
When this attempt failed, the committee began an intensive 
study of terms and concepts in general use Lists of important 
words taught m various courses were combined into a master list 
and teachers at all colleges asked to indicate whether each term was 
taught explicitly and to suggest the degree of importance attached 
to It The seventy five most frequently used and most important 
terms were then developed into objective test questions The 
resulting test was tried out. the items revised, and a final form pre 
pared ^ 

The Items on this test do not represent any radical departure 
om Stan ar testing techniques The greatest innovation of the 
r^nVl 1“, “hich the instrument reflects cur 

shown h^t in most humanities courses The items 

shown below are examples dratvn from th.s final form of the test 
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2. A sail boat. 

3. An electric lamp that looks like a candle. 

4. An office desk made o£ metal. 

5. A carving knife with a wooden handle. 

Critical Analysis and Judgment in the Humanities 
By far the greatest interest of the Humanities Committee cen- 
tered in a project designed to investigate a concept which ^vas 
called "critical analysis and judgment." While not overtly taught 
for in many humanities courses, objectives of most courses claim 
skill in such an ability as a major goal. Thus, at a verbal level the 
stage was set for a detailed inquiry into the nature of critical 
analysis and judgment and the degree to which students developed 
facility in it as a result of general education course experience. 

The concept of critical analysis and judgment is an attempt to 
reconcile two somewhat divergent theories regarding the nature 
of humanistic study. One point of view holds that humanistic 
works can and should be studied with the same methods and identi- 
cal rigor with which the objects of scientific investigation are re- 
garded. The only variable is the quality of the object of study. The 
botanist regards a leaf in its separate parts and in its unity. Based 
upon what he sees, he makes inferences and invents hypotheses 
which he tests in any of a variety of ways. In the same manner the 
social scientist regards a group as it functions, and from his observa- 
tions he seeks to find explanations and laws. By similar application 
of the methodology of scientific investigation the various aspects of 
awork of art may be analyzed to obtain greater understanding of the 
laws which operated to bring it into existence. If a painting is the 
subject of investigation, the uses to which the primary raw material 
and artistic media are put can be noted and recorded. The subject 
matter or content of the picture, the technical proficiency of the 
rendition, the various uses of line, color, and shadow, indeed the 
total form of the work, can be made the basis for inference and 
generalization. 

Viewed in this light, the skills necessary for the study of works 
of art are precisely those involved in critical thinking in the social 
or natural sciences. Indeed, at one stage in the work of the Hu- 
manities Committee the skills which its members believed essen- 
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tial for critical analysis were compared with those developed by 
the Critical Thinking Committee and were found to be almost 
identical 

However, most teachers of the humanities would reject this 
theory if it stopped there They would contend that such a view 
violates the essential subjective quality of the humanities Many 
teachers feel that critical analysis or the emotional feeling for a 
work of art are but means to the end of careful, mature assertions 
o subjective value judgments It is this end which makes humanis 
tic study unique, for the sciences or social sciences do not generate 
inquiry for the ends of pleasure, enjoyment, personal satisfaction 
or est etic appreciation Some teachers, however, would even 
reject critical analysis whether or not judgment were made the 
analysis They would feel that an individuals 
enrp wh k ^ should be an emotional, aesthetic experi 

levpl Tn^ work IS approached on an intellectual 

mieht hp^ ^ I concept of critical analysis and judgment 
uvfsts ‘he pure objec 

neither she ° subjectivists, for it attempts to emphasize 
neuher sheer critieism nor sheer emotive response 

meaninvnr'^^f"'^'^ ***« committee to define its 
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■natiat J-'dfi-ent Test 

most part to demand rnn, ‘he 

some demonstration of i' knowledge of specific content, 

short poems Students wMe LkL to ’“"T ™ 

of the thoughts of each poem suS « comparnons 
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Essay questions obtained were of the type: 

Compare the Parthenon and Chartres Cathedral as architectural 

expressions o£ the Greek and medieval attitudes. 

Since the same sort of test materials were to be found in the 
various published tests examined, it became apparent that in spite 
of expressions to the contrary, representative teaching and evalua- 
tion in the humanities were restricted to knowledge of fact with 
some incidental attention paid to intellectual manipulation of that 
content. The implication was obvious that a test of critical analysis 
and judgment would be a somewhat new development requiring 
its own unique methodology. 

Turning to the actual study of specific works of art, the com- 
mittee attempted to specify what critical analysis and judgment 
involves by demonstrating the skills which each member would 
employ in a critique of a work of art. In a sense the group was 
attempting to develop a theory of criticism appropriate for fresh- 
man and sophomore college students. This system, after consider- 
able experimentation and revision, was finally crystallized as a List 
of Abilities in Critical Judgment in the Humanities. It contained 
three major parts demanding (1) a subjective reaction, (2) analysis, 
and (3) critical judgment. Under each major heading are elements 
or abilities comprising it. Each ability is couched in sufficiently 
general terms as to be applicable to any of the major art media. 

A LIST OF ABILITIES 

IN CRITICAL JUDGMENT IN THE HUMANITIES 
Part One: Informal Notes 
Immediate (Pre-Analyltcal) Subjective Reaction 
1. The subject matter and idea of the work, and its expressive, com- 
municative, and evocative roles. 

a) Recognition of the subject matter or explicit recognition of the 
fact that there is no subject matter in the representational, imita- 
the, or referential sense. 

b) Statement of personal thoughts and feelings of the student 
evoked by tlie work. 

c) Explicit recognition by the student of the subjectivity of per- 
sonal tlioughts and feelings evoked in him by the work. 

d) Comment on tlie probable ability of tlie work to arouse sub- 
jective reactions in people other than ■ 
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f) Description of ideas and feelings of the maker which seem to 
be expressed in the work 

Analysts 

2 Function and Context— The action and roles of the work of art in 
relationship to the various combinations of circumstances (contexts) 
within which it has had or does have meaning 

a) Recognition of any political, religious, sociological, or economic 
functions of the work, or of the fact that it has none 

b) Recognition of the esthetic funaion of the work, its role in 
bringing pleasure through perfection of form 

c) emonstration of the influence of various external factors, such 
« social context and environment, on the nature of the work 

) peculation on the influence that the work might have upon 
society ° 


* 1 ,^ nature, use, and importance of medium in the work 
medium is used to mean language m literature, 
music, as well as medium in painting 
descnption of the medium made use of in the 

^ description of the general methods of expres 

and Tmw ^ Guide to Critical Analysis 

fNot annf™ M Humanities)* made use of m the work 

^ot applicable to music) 

medi^'*^!!»°^ conventionality or unconventionality of the 
expression ° ^"fluage, use of instruments, or method of 

the raed?iim” limitations and possibilities of 
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the maker ^^^letic problem influenced 

f) SHlufto “ P^rucul^ medium 
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major parts "whose ^llt of separate 

cance of the form of another create the signifi 
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o) Discovery and isaI ? 
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(' devices’ in characterization of the major elements 

e) Comment on the evA,.i 

'ices’ in literature) ^ quality of separate elements (‘de- 
'S« pp 157-58 



OBJECTIVES IN THE HUMANITIES 155 

d) Comment on the independent function that separate elements 
("devices" in literature) have in the total work, or recognition 
that there is no independent function. 

e) Recognition of symbols or symbolic usages together with de- 
scriptions of their use and meaning in the work. 

f) Recognition of the conventionality or unconventionality of the 
use of elements ("devices" in literature). 

g) Recognition and description of a plan, scheme, or structure on 
which the work is based— a principle of organization governing 
the relationship of formal elements to each other in the total 
work. 

h) Recognition of qualities or aspects of organization (without 
necessarily naming them as such) such as the following: domi- 
nance, suljordination, unity, variety, balance, symmetry, propor- 
tion, continuity, consistency, emphasis, tension, rhythm, move- 
ment, repetition, distortion, etc. 

i) Exemplification of qualities or aspects of organization found in 
the work by concrete references to the work. 

y) Speculation as to whether external factors have influenced the 
nature of the organization of the work. 
k) Comment on the use of organizational devices or over-all organi- 
zation for expressive purposes. 

5. Style.— Qualities of the work which are characteristic of the various 
forces which have affected its production. 

a) Description of style characteristics generally typical of the 
historical period or era. 

5) Description of style characteristics of the work typical of the 
particular school, trend, or movement within a historical period, 
c) Description of style characteristics of the work typical of the 
maker of the work. 

Part Two: Criticai. Judgment 

6. Part Two offers the student an opportunity to express a total re- 
action to or judgment of the work. Presumably he will make use 
of ideas or facts perceived during the analytic process on Part One, 
but he is free to select, reject, and add as he wishes. His statement 
should sum up to his concept of the general meaning or significance 
of the work. While this statement should be an over-all reaction, 
including the factors concerning the work that he thinks are most 
pertinent, it cannot be expected to cover and include all of the 
various kinds of significance which the work has within its total 
field. 

a) Demonstration of a total, over-all coherent perception of the 
significance of the work. 
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b) Perception of the interrelations of form and content in the 
work 

c) Evidence that the student has attempted to respond to the 
work at its primary level of significance, verbal, visual, sonic, 
tactile, etc 

d) Effective use of factors noted in Part One, evidence of care in 
both inclusion and exclusion 

e) Evidence of an attempt to express reactions other than logical 
or rational 

f) Demonstration of the ability to separate personal, subjective 
judgments from judgments based on qualities intrinsic in the 
work 

g) Recognition of the various contexts within which value judg 
ments can be assigned (comparison of this work with other 
works by the same artists with works of art of all kinds and in 

times with important human endeavors in any field) 

} Recognition of the unique quality of the particular work and 
paru^T^^ fixture of the student's experience in perceiving this 
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were prepared for painting, music, literature, and philosophy. 
Each series culminated in a broad requirement for a one-page 
critical essay expressing a judgment of the work of art. Each set of 
questions was printed on a form with space provided below each 
question for students to write their answers and a space on the 
back of the form for the essay. All forms are similar to the items 
for painting presented below. 

GUIDE TO CRITICAL ANALYSIS AND JUDGMENT 
IN THE HUMANITIES 

Part One: Informal Notes 

In Part One you are asked to make notes in the spaces provided. 
Formal sentences are not necessary. There will probably be more 
things to note under some headings than others, but you should 
make your coverage as complete as possible. You will be told when 
the time allowed for Part One is up. 

Subjective Reaction 

1. Make notes on what the painting is about (subject matter), on how 
it affects you, on how it might affect other people, on what the feel- 
ings of the painter seem to have been, etc. 

Analysis 

2. What seem to be the creative and/or technical problems (matter of 
function, purpose, form, space limitations, etc.) with which he is 
dealing? 

3. Describe the general method of expression (handling of space, dis- 
tortion, use of light, interest in textures of the objects represented, 
etc.). Comment on any relationship you can detect between the re- 
sults obtained and the medium used, and the appropriateness of the 
clioice of medium. How did the artist's problems influence his choice 
of medium? 

4. Describe the elements of the painting (things which can be con- 
sidered separately, such as line; color, including hue; intensity and 
use of light and dark; shape; texture of paint; etc.). How are the 
elements used in the organization of the painting (specific devices 
or plans used to hold the elements together, or the lack of such plans). 

5. Indicate the characteristics of the painting that lead you to believe 
that it belongs to some particular period, school, or artist. 

Part Two: Synthesis: Expression of Over-all Judgment 

6. In Part One you have made informal notes on the picture from 
various points of view suggested to you. In this part you are asked 
to wite your over-all reaction to and judgment of the painting. 
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Organize your ■\mting Use only as many of the ideas jotted down 
in Part One as you think are pertinent, plus any additional ideas 
)ou may have Your final judgment should represent much more 
than a compilation of the notes you have made The important thing 
IS to express in a good well rounded essay your judgment of the 
general significance and meaning of the painting 

Evaluating student performance on these Guides to Critical 
Analysis and Judgment in the Humanities involved comparing 
Items from the List of Abilities with student responses Credit was 
to be assigned for each ability demonstrated, thus considerably 
more reliability of scoring was ensured than could usually be at 
tamed with more conventional essay responses However, evaluat 
ng res^nses proved much more difficult in practice than in 
eory areful reading of student papers in a search for previously 
T*. ^ difficult, tedious and time consum 
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a curious attitude toward the objective test-essay issue. Projects 
begun with the creation of an objective test ended with an ap- 
proach to the same ends by means of an essay test. On the other 
hand, projects initially concerned with essay responses have turned 
almost irresistibly to explore objective-test possibilities. The hu- 
manities project has been no exception. After a rationale of criti- 
cism was established, attempts were made to test for the same 
abilities by objective or short-answer tests. While these attempts 
were not notably successful, one technique is of sufficient impor- 
tance to warrant attention. 

At one institution a series of pictures has been prepared in a 
manila folder, one set for each student. Then, separate objective 
questions were prepared to sample the abilities from the List of 
Abilities in Critical Judgment. These questions may deal with 
from one to five of the pictures and the folder contains enough 
different prints to allow considerable flexibility. One form, for 
example, contains the following pictures: 

Daumier, The Washerwoman 
Lippi, Madonna and Child 
Vermeer, Lady with Lute 
Murillo, Children of the Shell 
El Greco, The Holy Family 
Matisse, The Blue Window 
Bivera, Mexican ChjJd 

Typical questions are: 

1. In which of these does the artist seem less interested in the 
subject than in formal problems? 

2. Which of these seems to you more sentimental? 

3. Which uses a subject which has a Biblical or mythological 
source? 

4. Which picture uses most cool colors? 

5. Which is more dramatic in its use of highlighting? 

6. Which picture has the center of interest farthest to the left? 

7. Considering the backgrounds of these paintings, whicli is the 
earliest in manner? 

Research Results 

The several tests or inventories prepared by the Humanities 
Committee have been used in several kinds of research. The first 
kind consisted of studies made of the instruments themselves to 
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appraise their reliability, internal consistency, and validity The 
SKond sort has consisted of using the instruments as means of 
obtaining information about the humanities m general education 
As has been indicated earlier, the Attitudes Inventory did not 
prove successful The items proved so easy that an inadequate 
range of scores was obtained to discriminate even between indi 
vi ua s possessing known differences of feelings concerning the 
umanities The small range of scores skewed toward one extreme, 
resulted m an excessively low estimate of test reliability 

1 he Humanities Participation Inventory, however, has proven 
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that commonly used. Further, teachers in the humanities have 
generally been suspicious of any kind of evaluation other than 
content-centered essay testing. It was therefore quite difficult, and 
at some institutions impossible, to interest humanities teachers, 
other than those actually on the committee, in the instrument 
Many rejected it a priori as inadequate either because it did not, 
in their opinion, tap analytical abilities, or because it did not in- 
volve knowledge taught in a particular course. However, the 
guides ivere used e.\tensively at a few schools. Our opinions of 
tile instrument have been formed largely from their results. 

Papers from these schools, dealing ivith the same art object, 
were pooled, reproduced, and scored by a number of raters. Tables 
28, 29, and 30 present the coefficients of correlation between the 
ratings assigned by different raters. These figures are much higher 
than those usually obtained from several raters judging students’ 
written work. They take on increased significance svhen it is 
realized that the raters came from different institutions approach- 
ing the humanities from several different points of view. 

Estimates of the validity of an instrument such as this one are 
difficult to obtain. Indeed, the ability involved in the guides took 
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Music and Literature forms showed means which were statistically 
significant (1 percent level), the same direction of differences for 
painting reflects the main point— that the instrument distinguishes 
between these groups. 

Since the Vocabulary Test was completed only toward the very 
end of the Cooperative Study, few data have been collected be- 
yond item analysis data and estimates of reliability. Both kinds 
of evidence were satisfactory. Only three out of seventy-five items 
yielded low coefficients of discrimination. The reliability of the 
test was estimated at .83. 

The Participation Inventory assesses a trait which it is quite 
logical to assume is but slightly related to academic aptitude or to 
tests requiring highly intellectual skills such as the Test of Critical 
Thinking. Evidence in support of this has been obtained from 
every study made. Both pre-test and post-test results have been 
compared with other pre-test and post-test scores for students 
from several colleges. The results of these comparisons, expressed 
as correlation coefficients, are given in Table 32 below. 

The total participation scores of 100 students were correlated 
respectively with Art, Music, and Literature sections from the 
Kuder Preference Record— Vocational, with r s of .08, —.12, and .17 
resulting. From these data it would appear that the trait being 
measured by the Participation Inventory is considerably different 
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from the preference ratings achieved by the use of the Ruder test 
The Participation Inventory scores correlated quite diversely 
with scores made on the various forms of the Guide to Critical 
Analysis and Judgment in the Humanities, as is demonstrated in 
Table 33 

Another senes of correlations were made between scores on the 
Humanities Vocabulary Test and each of the three levels of par 
ticipation-frequency. enjoyment, and senousness-both on a pre 
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level of participation (frequency, enjoyment, and seriousness) to 
the five major areas of the arts. The letters to the left in the table 
indicate school, and the numbers on the same side of the table 
represent the degree indicated for each level of participation. For 
example, 15 percent of the students at School A indicated no par- 
ticipation in literature, 59 percent claimed some participation„and 
24 percent indicated considerable participation in this area. In the 
Combined Arts, of those indicating some participation, 1 percent 


TABLE 34 

Pasticipation or Students in Foun Colleges in Five Major 
Areas of toe Humanities 



did not particularly enjoy it, 20 percent enjoyed it fairly well, and 
62 percent enjoyed it a great deal. 

These data may be studied more effectively if combined into a 
shorter table. Table 35 presents averages of percentages of stu- 
dents who responded to the frequency, enjoyment, and seriousness 
categories at each of the three levels of intensity. Thus the first 
column. Literature, indicates the percentage of students who par- 
ticipate at each of the levels iriih the specific category of frequency, 
enjoyment, and seriousness. From tin's tablesevcral generalizations 
emerge. 

(1) Students from each of tlic four schools participate at the 
lowest level in tlie areas of music and tlic plastic arts. (2) Students 
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TABLE 85 

Participatiov or Stttlents in the Humanities as to Frequenct, 
Enjotmbnt, and Seriousness 
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ranging from the one making the lowest initial score to the one 
making the highest initial score. Gains over a year's period are 
shown for each such school group. 

From these data the fact emerges that, in general, the lower the 
pre-test score, the greater the gain made during the course of a year. 
One might explain this situation by arguing that the more inter- 
ested students are not likely to increase the amount of participa- 
tion— although it is hoped that the quality may be modified. On 
the other hand, those with low records of participation before 
coming to college might be expected to increase greatly the 
amount if for no other reason than the fact that so many activities 
are available on a modem college campus. The data seem to indi- 
cate this is so, and this is given considerable support by a study of 
changes in participation with respect to each activity made by stu- 
dents from one university over a period of a year. The following 
generalizations are based on data from that college. 

1. More women apparently read novels and short stories before 
coming to college than did men (65 percent to 49 percent answered 
“frequently”). Both men and women read novels less frequently 
at the end of the year than at the beginning. Possibly the greater 
amount of nonfiction reading demanded by college courses is the 
explanation for this shift. However, precollege men read with 
considerably more seriousness than do women. The relative posi- 
tion of the two sexes was reversed at the end of the year in this 
regard. 

2. Students read much less poetry than they do novels or short 
stories. Here again, however, the women have a better record than 
men. And again women develop greater seriousness in their read- 
ing of poetry than do men. Only a few students, however, regard- 
less of sex, believe they read poetry at the highest level of serious- 
ness. 

3. Apparently college makes very little difference with respect 
to students* writing short stories, poems, and essays. Only a few, 
regardless of sex, claim to do so frequently. Men, however, gain in 
enjoyment in doing so over a year, whereas women remain about 
tlie same in this regard. 

4. IVhen students come to college and at the end of a year of 
college, tliey attend movies frequently and with great pleasure. 
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Their college experiences do not seem to result in any more sen 
ous approach to such participation 

5 Students do not attend plays frequently before or after a year 
of college Nor do those who do attend do so with appreciably 
greater pleasure or seriousness Men. however, drop in the fre 
quency with which they do this activity 

6 No marked changes appear to take place with respect to 
attendance at operas and ballets Students are about evenly di 
vided between those who never attend and those who attend occa 
sionilly (which they define as once or twice a year) 

As could be expected students like to and do, listen to the 
radio and television both before and after a year of college 

tu ents increase rather markedly the frequency with which 
m plays, skits, and the like The enjoyment which 
activiiT^^^ seriousness with which they approach these 

activities do not seem to change 

concern attendance at 

“ classical music 

students nia slight drop in the frequency with which 

cieaseTnfhr, ‘here is a similarly slight in 

«ease in the seriousness with which those who do play tLir instru 

aheL*yeatorco1l’cge°‘ 

prints, and they Increa”'^il*^ attend art exhibits or look at art 
year— men more sn tU 'andency to do so over a period of a 

mcreases, ar^gh^ 

courses available to them^ orientation of the 

to painting from^thTbc the same with respect 

sarnie pro4 hoTds' t^e ^.h r^^e ft' T 

students do and about half fl,, handicraft About halt the 

14 There is a slieht m '"gage m such activities 

dents read seriously^and emoTuh 1 ^" with which stu 

are considerably lek than hiL^ '“sophical essays, but the gams 

same sort of profile holds true witf"” "'‘S''t wish The 

philosophical and theological topics"'*""' d'toossion of 
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These data, drawn from one institution, do not indicate that 
much gain is being made in the amount or level of participation in 
the fields of the humanities. Nor could such gains reasonably be 
expected in view of the way college humanities courses are organ- 
ized. As a matter of fact, one analysis of gains made by three 
groups of students— (1) those having had general education hu- 
manities courses, (2) those having had liberal arts courses in the 
humanities, and (3) those having had no course experience in the 
humanities— shows that the last category makes approximately 
the same order of scores in participation as students having general 


TABLE 87 

Hvmanitie3 Pabticipation Scores tor One Institution 


Group 

N 


8JD. 

General education humanities course 

61 

7i 98 

14.05 

Humanities courses other than general 
education 


90 08 

17 25 

No humanities experience 

S5 

74 44 

20.80 


education course experience. The markedly higher scores for the 
students taking non-general education humanities courses are very 
likely the result of selective factors such as including the scores 
of humanities majors. These data, indicated in Table 37, if char- 
acteristic of any appreciable number of colleges, suggest that some 
rethinking of the role of humanities courses is in order. If these 
data are valid, they would suggest that the objective of increasing 
participation either is inappropriate or that considerably more 
should be done to implement it. 

It is unfortunate that the project upon which the Humanities 
Committee expended the greatest amount of time and effort should 
be supported by the least amount of empirical data. However, the 
very lack of such information is rather definite evidence about the 
activities and products of liumanities courses. 

The original research design called for students at the various 
colleges to view tlie same painting, read the same selection, or 
hear the same composition, both in a pre-test and in a post-test 
situation. The tvwks of art selected were (1) El Greco, Vteio of 
Toledo, (2) Shakespeare, Sonnet XXX (“When to tlic sessions 
of s^^cct silent thought. . . -**), and (3) Prokofiev, Classical Sym- 
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phony, First Movement The test was administered m the fall of 
1951, but the scoring of the test proved most difficult for staff 
members asked to participate in the project 

1 The idea of appraising quantitatively students’ responses 
which they believed should only be appraised qualitatively was 
distasteful to many teachers m the humanities Refusing to accept 
the assumption that quality of analysis and judgment could be 
inferred from quantity, such teachers rejected the rationale of the 
grading system, and thus invalidated results for research purposes 

2 The idea of restricting ones evaluation of students papers to 
preconceived elements instead of evaluating in ter ms of subjective 
intuition also proved to be a barrier Teachers of the humanities 

e leve generally that estimates of student demonstration of cnti 
cal analysis should be made by teachers unfettered by any particu 
ar set of categories on responses on which they can base a grade 
n a sense they believe each student paper is as unique as a work 
judged by criteria which appear to the initiated 
rmtr/ paper If this premise is accepted then of 

^ ^uti^ rationale of the Guide to Critical Analysis and 
ratinm^^^ ^ ^ Humanities must be rejected However, the 
» rejected^ accepted m theory It is only the application of it that 

and rneamnl against the guides is ambiguous in nature 

the truides ^th ^ound that students did not achieve well on 

sLiSdiL' n' ''^"'onstTBted a rather marked inability to 
inferred rhit th"” From this a number of teachers 

exposed to hi ^ ^ invalid because students who had been 

formancrand lo rerchThe ” evidence of nonper 

not the instrument P«haps‘'srair‘'’'' 

of the wav a nurnl,,, r u ^ ^e expected in view 

tions of classes anrt ““''^es are organized Observa 

in typical hunianilies"coms« rad* 

tlicn come to class cither to , fr°na textbooks and 

ntoti, some work of arro m'l 

r to listen to or see a work about which 
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they have studied. In either event they are acquiring knowledge 
but have little experience related to other objectives. 

If the objectives of increased participation, analytical ability, or 
genuine appreciation are valid, then the way many humanities 
courses are conducted must be judged inadequate. 

These are broad generalizations. Perhaps a few descriptions of 
actually observed class situations might underscore the differences 
between stated course objectives and classroom practice. 

In one class, organized along cultural history lines, an hour was 
devoted to hearing three students read essays about artists from a 
particular period. The assignment, made several weeks earlier, 
was for each student to select an artist and do a research paper on 
him. These papers were then to be read in the class, which con- 
sisted of twenty-five students. At the rate of three or four a day, 
two weeks of class time were thus utilized in this activity. Un- 
fortunately, the potentialities of student reporting were lost be- 
cause no time was available for class discussion of the points raised 
in each paper. The reader was putting on a performance for one 
person— the teacher. The emphasis in each paper was on the fects 
of the painter’s life— what analytical comments were included came 
obviously intact from some source book. 

At another institution the humanities course was also of the cul- 
tural history type. Some two hundred and fifty students were 
gathered in a large lecture room to hear a professor read a paper on 
the beginnings of the Renaissance. The lecture was intended to 
provide a knowledge basis fora week’s study in smaller classrooms. 
In the smaller sections, teachers gave other lectures covering in 
more detail the same ground broken by the speaker for the week. 
At no time were students encouraged to do anything other than 
respond to questions, the answers to which were to be found in a 
textbook or lecture. 

In still another institution in which the course was of the criti- 
cal-esthetic type, students were gathered in a small room while the 
teacher played selections from Bach and Beethoven and carried on 
a running commentary about them. The teacher’s remarks were 
sound and he sought by a variety of ways to show many implica- 
tions in other fields for the work of the composer. The teacher 
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demonstrated a general education, but he was giving the students 
the results of his analyses rather than providing opportunity for 
them to arrive at their own judgments 
These teachers were all sincere, hard w orkmg scholars who be 
lieved in general education, but one must nevertheless doubt 
whether by such procedures they could hope to change attitudes 
materially, increase the extent and seriousness of participation, or 
the critical facilities of students Then why are such uninspired 
activities continued? A subjective judgment suggests the existence 
of three factors First, a minority of teachers are convinced that 
out of broad knowledge will develop— for some students, at least- 
all the desired outcomes A second factor is that the humanities, 
perhaps more than any other general education area, are beset with 
the complex problem of selection of the most significant contribu 
tions of all the ages to man s present cultural heritage The vast 
ness of the area and the often pronounced differences in viewpoint 
among the members of a humanities staff result in courses which 


are so crammed with important materials that mere coverage of 
them IS a difficult achievement Finally, instruction aimed at ob 
jectives such as aitical judgment poses difficult and unsolved 
clearly seen by a brief review of the experience of 
the Humanu.es Committee m developing the guides 

e ui es to Critical Analysis and Judgment m the Humani 
les, espite the devoted and continuing effort of a group of hu 
manuies teachers, are but rough first attempts to measure a skill 
M humanities have done but little to develop 

fonn they are difficult for students to use and 
Tr" "^“hers to grade They also fall 

thesp fl fr ^ committee envisaged it In part 

conunue^'en “ ''hich could be overcome by 

the ditTiri li 1°^' “tnplex. however, is the fact that some of 

d nui ih ‘ of teachers and stu 
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fundamental thinking of the type reported in this chapter can 
much change be expected. In saying this, we do not suggest that 
others will arrive at the same conclusions or the same instruments, 
but if the work on the Guides to Critical Analysis and Judgment 
in the Humanities, for example, can provide leads for other kinds 
of evaluation and instructional activities, it will have been very 
worth while. 



7 

PERVASIVE OBJECTIVES 1: 

CRITICAL THINKING 


Intercollege Committee on Critical Thinking Lily Detchen, 
Pennsylvania College for Women, Wesij:y Dykstra, Muskingum 
College, Harvey Gelder, Western Washington College of Edu 
cation, Walker Hill, Michigan State College, Kellogg W 
Hunt, Florida State University, Morris T Keeton, Antioch Col 
lege, Cecil H Miller, Kansas State College of Agriculture and 
Applied Science. W Edgar Moore. University of Florida Troy 
Orc^, Pennsylvania College for Women Carl N Rexroad, 
Stephens College, Sylvester R Toussaint, Colorado State Col 
lege of Education 


T he very existence of the Intercollege Committee on Critical 
Thinking at once bears witness to the generally accepted 
importance of the objective and to the strong disagreements 
^ to Its nature and attainability Despite the idealistic basing of 
le SIX committees on objectives, it is not unfair to say that, prac 
a y spea mg four of them arose primarily out of concern about 
e general education content areas usually designated as commu 
science, and the humanities This was not 
r^*m remaining two objectives Although con 

rnitrsp«°^ cntira thinking had led to the organization o£ special 
nrnrt t^olleges, this was atypical The more usual 

« an objective to be achieved through 

bu^r T - -P^^te committee 

convmced nHh f Those 

1 ^ ^ of the primacy of such a committee 

liad two major arguments (1) that general education courses to 

174 
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date had not emphasized critical thinking and that, accordingly, 
there was a strong possibility that other committees would ignore 
the objective; (2) that critical thinking stated as a general educa- 
tion objective is not limited to course content but rather is re- 
garded as a behavior desirable in all the activities and problems in 
which the individual becomes involved. 

Added to these arguments were the voices of one or two persons 
representing institutions with courses in philosophy, logic, or 
"clear thinking" which would othenvise not be represented. 
American— and particularly college faculty-mores dictated the 
formation of a committee as the logical way of quieting the disa- 
greement. 

Underlying this issue of the need for a separate committee on 
critical thinking are some basic differences in viewpoint about the 
nature of the learning process. Those opposing or at the least 
doubting the advisability of a separate committee feared that it 
betokened a faith in the discredited formal discipline or faculty 
psychology. In general, it has been found that no learning experi- 
ence has any extensive mental disciplining power. Thorndike and 
others rather conclusively demonstrated that such transfer as does 
take place happens only when there are identical elements or 
components in the learning and the transfer situations. Arguing 
from this, it is plausible that the ability to think in science must 
be developed by studying science, and that the ability to think on 
social issues must be developed by a course dealing with social 
issues. Contributing to this viewpoint is the obvious principle 
that critical thinking about any issue must be based on the knowl- 
edge of relevant facts and generalizations. 

An opposing viewpoint accepts the necessity of identical ele- 
ments but points to some evidence that transfer does appear to 
take place to a greater extent and, in some cases, in a some^vhat 
different way, than is expected according to this theory. Transfer 
is apparently promoted by the meaningful organization of knowl- 
edge, by habits of work, methods of attacking problems, and by a 
developed motivation to seek for the transfer of knowledge and 
abilities. Added to this is a suspicion on the part of many con- 
cerned with general education that the theory of identical ele- 
ments and its reinforcement of the importance of specific knowl- 
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edge have resulted in the use of a formal discipline approach m 
many general education courses Knowing as a prerequisite to 
thinking IS all too easily misinterpreted to mean that thinking 
follows naturally out of knowledge— or, at least, that it should A 
logical extension of such reasoning is that cntical thinking is a by 
product of instruction rather than an immediate objective 
The preceding rationale of the differences in viewpoint about 
committee organization for attacking critical thinking was not 
explicit in the deliberations, but in retrospect it appears to have 
been the fundamental basis for disagreement The Critical Think 
ing Committee itself was constantly concerned about the issue of 
the pervasiveness or specificity of the objective It had high hopes 
that Its work, coupled with that of other committees, might throw 
some light on the problem Later evidence will show that this hope 
was achieved only if a sufficiently diminutive meaning is accorded 
to some’ 


The Objective of Critical Thinking 
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the instructor in his lectures, but this broad conception is useless 
for either evaluation or instruction. 

One of the earliest points of agreement about critical thinking 
rvas that it did not include “imaginative thinking/’ but this nega- 
tive approach was quickly abandoned for a positive approach in- 
volving a listing of critical thinking abilities. For this purpose 
“problem solving,” which was regarded as embracing most of the 
aspects of critical thinking, was adopted as the ability to be par- 
ticularized. This restriction was decided upon because of a feeling 
that problem solving is essential to effective living. A detailed list 
of problem-solving abilities follows. 

A TENTATIVE UST OF THE PROBLEM-SOLVING ASPECTS 
OF CRITICAL THINKING 

1. Ability To Recognize the Existence of a Problem 

a) To recognize related conditions in a situation. 

b) To recognize conflicts and issues in a situation. 

c) To locate “missing links” in a series of ideas or incidents. 

d) To recognize problems which have no solution, 

2. Ability To Define the Problem 

a) To identify the nature of the problem. 

b) To undentand what is involved and required in the problem. 

c) To recognize ways in which the problem can be phrased. 

d) To define difficult and abstract elements of the problem in 
simple, concrete, and familiar terms. 

e) To break complex elements of the problem into workable 
parts. 

f) To identify the central elements of the problem. 

g) To place the elements of the problem into an order in which 
they can be handled. 

h) To eliminate extraneous elements from the problem. 

i) To place the problem in its context. 

3. Ability To Select Information Pertinent to the Solution of the 

Problem 

a) To distinguish reliable and unreliable sources of information. 

b) To recognize bias upon which information is selected and 
rejected. 

c) To recognize information relevant to the solution of tlie prob- 
lem. 

d) To select adequate and reliable samples of information. 

e) To systematize information. 
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f) To select information from personal experience relevant to the 
solution of the problem. 

4. Ability To Recognize Assumptions Bearing on the Problem 

a) To identify unstated assumptions. 

b) To identify unsupported assumptions. 

c) To identify irrelevant assumptions. 

5. Ability To Make Relevant Hypotheses 

a) To discover clues to the solution of the problem. 

b) To formulate various hypotheses on the basis of information 
and assumptions. 

c) To select the more promising hypotheses for first considera- 

tion. 

d) To check the consistency of the hypotheses with the informa- 
tion and assumptions. 

e) To make hypo^eses concerning unknown and needed infor- 
mation. 


6. /Ifeility To Drav} Conclusions Validly from Assumptions, Hy- 
potheses, and Pertinent Information 

X® logical relationships among terms and propositions. 
\ y®^<5gpize necessary and sufBdent conditions. 

S, cause and effect relationships. 

a) To identify and state the conclusion. 

^oliclu^^ the Validity of the Processes Leading to the 

a) To distinguish validly drawn conclusions from others chosen, 
or example, because they are in accord with values, preferences, 
and biases. 

r) ^ inference from a probable one. 

ect formal logical inconsistencies in the argument. 

8. To Evaluate a Conclusion in Terms of Its Application 

conditions which would be necessary to verify a 

^ conditions which would make a conclusion in- 

'problem'^ si^cquacy of a conclusion as a solution of the 


inti- U ^ many possible. Critical think- 

snecifirt mental activity, and an almost endless list of 

comnletplv” i P^°duced, no two items of which would be 
indcDcndenf” Equally clearly they would not be 

independent, particularly if an attempt is made to view them as 



PERVASIVE OBJECTIVES 1: CRITICAL THINKING 179 

they might be applied in solving a problem. Therefore, this or any 
other list must be regarded primarily as an aid in understanding 
the nature of critical thinking, and the specifics of the list must 
not be allowed to usurp attention to the point where their integra- 
tion and augmentation by other unlisted abilities into the still 
undefined complex mental ability— critical thinking— is forgotten. 
The list of abilities above is not startlingly different from others 
that have been devised and no great claim for originality can be 
made for it. One bit of evidence on this point is that three other 
committees of the Study working independently produced lists 
which were essentially in one-to-one correspondence to the above. 
Evidence of the fallibility of the list is found in the extent to which 
it undenvent revision and reduction. 

A Brief List of Criticai. Thinking Abilities 

The list of critical thinking skills utilized by the committee in 
its later deliberations involved only five points. These, accom- 
panied by a brief rationale, follow: 

1. The ability to define a problem. 

People frequently attempt to settle difficulties without even 
knowing what the difficulty is. Training in identifying and 
defining problems may relieve much of the frustration gener- 
ated in such cases. The ability to define a problem is also im- 
portant in listening and reading. The trick of good reading 
and listening often is simply a matter of identifying the prob- 
lem dealt with so that the ideas fall into a pattern. 

2. The ability to select pertinent information for the solution 
of a problem. 

Once a person has defined a problem and has thought 
about it sufficiently to be able to state the problem in his 
own ivords, he needs to decide what kinds of information 
might help him in solving the problem. This involves judg- 
ing what kind of information is pertinent to the problem, 
and it involves estimating the reliability of information 
wliicli might be pertinent. These abilities are necessary for 
the intelligent solution of problems, and interact witli tlie 
ability to see what the problem really is. 
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3 The abtlity to recognize stated and unstated assumptions 
By an assumption is meant something that is a part of the 
argument but is taken for granted without any argument to 
justify It It may be stated, or it may not be If it is stated, it 
is simply said to be true, not shown to be true Probably the 
speaker (or ivriter) expects it to be accepted without any 
question Sometimes an argument rests on an assumption 
which IS not stated at all, in feet, the person making the 
argument may not be aware that it is involved at all But it 
must be believed if the argument is to be accepted 
This IS not to say that we should never make any assump 
tions when we form conclusions on the basis of evidence We 
can t avoid making assumptions But we do need to be on the 
lookout for those assumptions (in our oivn arguments and 
in those of others) which are important because they make 
a difference in whether or not the conclusion ought to be 
accepted 

4 The ability to formulate and select relevant and promising 
hypotheses 

It IS important to make a deliberate effort to think of as 
many possible solutions or hypotheses as possible and to de- 
velop the habit of doing this Then we examine the possi 
bihties to see where each of them would lead We think them 
through to their probable consequences Some of them may 
not require much thought, they can be quickly eliminated 
from further consideration Others may need careful exami 
nation before we decide which hypothesis seems to be the 
best one 

5 The abihty to draw conclusions validly and to judge the 
validity of inferences 

\\ hen we put pieces of evidence together and draw con 
elusions from them, it is important that we do it in such a way 
that the conclusion really follom from the evidence This is 
w at we usually call correct reasoning’ or ‘ logical think 
mg Granted that our evidence is pertinent and that it is 
rue, We can still make mistakes in judging what conclusion 
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it supports. When we avoid these mistakes, and reach a con- 
clusion that does follow from the evidence, we have a “valid” 
conclusion. 

We can learn to judge when we are confronted with a valid 
argument and when we are not. We particularly need to be 
able to judge those situations in which common beliefs and 
our own preconceptions might make us inclined to accept a 
conclusion that is not validly based on the evidence. Obvi- 
ously, this is true both rvith respect to conclusions which we 
form for ourselves and with respect to conclusions which are 
formed by others and which we are asked to accept. Along 
with the ability to draw valid conclusions from evidence goes 
the ability to judge the validity of arguments which we hear 
or read. 

The process of combination of abilities and the dropping of the 
subpoints under the major headings resulted from the committee 
members finding, first, that the more detailed list had served its 
purpose in bringing about a common understanding of the abili- 
ties and, second, that it was both unrealistic and impossible to find 
situations and develop test materials which clearly and unequiv- 
ocally' involved the separate abilities of the more disjunct list. 
The same difficulty applied but to a lesser degree to the abbrevi- 
ated list. 


Review of Related Tests 

It may be accepted as one of the maxims of evaluation that any 
individual or g^oup undertaking an evaluation task will find all 
prior attempts unsatisfactory in some respect. Scientific objectivity 
dictates, however, that this be a conclusion arrived at after careful 
deliberation rather than an assumption made in advance. So it was 
with the committee. 

A review of prior ventures in the area of critical thinking re- 
vealed the following instruments which appeared to merit detailed 
study: 

1. General Educational Development Tests, College Level (Educa- 
tional Testing Service) 

2. American Council on Education Psychological Tests (Educational 
Testing Service) 
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3. Diagnostic Aptitude Tests (Psydhological Corporation) 

4. Primary Mental Abilities, Thurstone (Science Research Associates, 
Inc.) 

5. Yale Aptitude Tests (Educational Records Bureau) 

6. General Aptitude Test Battery, USES GATE (United States Em- 

ployment Service) 

7. Miller Analogies Test (Psychological Corporation) 

8. Warren R. Bailer, The Case of Mickey Murphy, A Case Study In- 
strument for Evaluating Teachers’ Understanding of Child Growth 
and Development (University of Nebraska Press) 

9. Watson-Glaser Test of Critical Thinking (World Book Company) 

. ests on Critical Thinking of the Progressive Education Associa- 
tion Eight-Year Study: Interpretation of Data. Application of the 
^inciples of Logical Reasoning, Nature of Proof (Educational 
Testing Service) ^ 

11. ‘«ts such as those in Thouless’ How to Think Straight 

(Simon 8: Schuster), and in various logic texts 

One or more faculty members in the various cooperating insti- 
m to review and analyze one of these tests by 

nir on"?® ' rendering an introspective report of 

d ms to IT'T (2) "tanging for stu- 

action to / then report on their re- 

tions procedure with the test items. These reac- 

questions _ “ ’“™"'"‘ted by a written response to a number of 

in ‘he review of a^e«mse«lnoTams'’t“ " 

2. Are the items''eaui'tahl"^ taking the test clear and understandable? 
'vhich the test purporrm me^lTre?® “"=8°ties of skills 

edge or subjcct^matf^^ spread among the various divisions of knowl- 
cerned? which the test purports to be con- 

^ the lime spccified'for t^uld be expected to finish the test within 

items? from fairly easy to fairly difficult 

student sufficiently varied tasks so as to maintain 

8. Docs the total tesTappear'io b'*™'n 

at uhich it will be used? * suited tor the educational level 

9. What is your general reaction to this test? 
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Utilizing such questions, extensive reviews of the instruments 
listed earlier were prepared. As expected, no instrument in its 
entirety was found to be satisfactory to evaluate critical thinking 
as construed by the committee, although particular items and 
testing techniques were found to be suggestive. While many par- 
ticular criticisms were made of certain instruments, the major 
shortcomings were these: (1) use of problems or tasks devoid of 
realism, such as puzzles or abstract numerical or geometrical prob- 
lems; (2) too limited coverage of critical thinking skills as listed by 
the committee; and (3) unsuitable content loading. The experience 
of reviewing these materials was productive in helping to crystal- 
lize the criteria to be met by any new instrument. The experience 
was also valuable in assuring the committee that what they envis- 
aged as critical thinking was not identical with intelligence or aca- 
demic aptitude as defined in tests of such abilities. 

The Specifications for a T3EST of Critical Thinking 

Coincidental with the review of existing instruments, attempts 
were made to collect accounts of situations illustrative of the criti- 
cal or uncritical thinking of students in respect to the specific abili- 
ties agreed upon. These situations were culled from test experi- 
ences and classroom, campus, or other situations. In some cases, 
students were invited to contribute from their own experience or 
from observation. The faculty member collecting the situations 
was also invited to attempt to formulate them into a series of test 
items. Questions arising out of the attempts to fulfill this appeal 
forced the formulation of specifications for the types of problem's 
or situations to be included in the contemplated test. The follow- 
ing criteria were arrived at: 

1. The situations should be realistic in terms of student experience— 
not hypothetical, but recognizable as actual in their experience or 
in the experience of people known to them. The possibility of 
involvement of emotional reaction or of attitudes should not in 
itself be the basis for elimination of a situation. 

2. The situations should be of such non-provindal nature as to be 
usable by all the institutions. 

S. The situations should be so selected that the evaluation device may 
be used over a period of time without being invalidated through 
emphasis on matters of transitory importance. 
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4. The situations should be independent of particular courses and 
content so as to avoid overlapping with special area projects and to 
make some provision for evaluating the total growth of critical 
thinking resulting from all courses. 

5. The situations should be so devised as to present all pertinent 
information or be so qualified that any information not so pre- 
sented may be expected to have been attained by most students. 
The vocabulary should likewise be within the range of the majority 
of students. 

6. Directions should be clear and less complicated than the thinking 
processes tested. 

7. Items should be kept free of the terminoloey and conventions of 
formal logic. 

8. For^oups of items having a common set of responses, the elements 

0 ^ sj™ple enough to be readily kept in mind. 

^ omp icated problems or situations involving a large number of 
inierre aied items should be avoided in favor of separate test situ- 
auons involving only one or two items. 

’ twelfth standard word list for the 

intent (if such exists) of using 
artificial or generally unfamiliar words. ' 

^ ^ regarded as necessary for a test for general 

miohr nfftn that issues of timely and local interest 

lackintr tl ^ instructional purposes than problems 

some snfri?^ r considered desirable to make 

tions .JughrL' Tn>wn''Vhe ton 

was f .u- ‘ ^ analysis of problem sources 

than exfcmstive. '* P“‘'P°^'- The subpoints are illustrative rather 
I- Problems in reference to self. 

rapabilitiB a^d 'r™* his own intellectual and physical 

tohing them? hm.tat.ons. how does the individual go about 

2. Ptoblcms in reference to other people. 

student procee^d'^lo*?n°t'^*”® motives of others, how does the 
pothescs about the moth?' 

strated behavior? Does he o' “'>='^"‘'"8 their demon- 

Gisen certain motives do* L , '‘""““ons of such hypotheses? 
present? ' ^ decide why such motives are 
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b) Presented with the facts of other persons’ values or individual 
points of view, how does the student proceed to interpret them? 
Is he able to see assumptions upon which such points of view 
are based? Can he predict inferences or implications which might 
be drawn from such points of view? 

c) Presented with the existence of inconsistent behavior on the part 
of individuals, is the student able to analyze the different roles 
in which they are acting? 

3. Problems in reference to the physical and natural universe. 

a) How does the student go about explaining the changing and 
unchanging elements of the universe? Does he attempt to bring 
what he knows about the universe to bear in solving his prob- 
lems? Is he, for example, aware of the implications for himself, 
as a man, of accepting Einstein’s theory of relativity? 

4. Problems in reference to society. 

a) How does the student think about differing cultural values? Does 
he engage in rationalization or does he seek to find explanations? 

b) Faced with various historical facts indicating social change, is 
the student able to see the implications of those facts? For ex- 
ample, can he accept the fact that the traditional family might 
be changing and might possibly disappear? How does lie explain 
the changes which have taken place in Western civilization since 
1800? 

5. Problems in reference to values and ethical standards. 

a) Can tiie individual think dearly and objectively about concepts 
of democracy? Can he see the implications of those concepts? Can 
he detect upon what assumptions those concepts are based? 

b) To what sort of critical analysis is the student willing and able 
to subject various systems of ethics and morality? At what point 
in the process of his thinking does he rely upon faith to sustain 
his point of view? 

These five areas were regarded as defining the “content” axis 
for a two-way chart describing the dimensions of possible evalua- 
tion instruments. Figure G illustrates the relationship between the 
two elements. The cells of the chart show the relative emphasis 
placed on a particular ability in reference to a particular content. 
The non<ontcnt column G was introduced to cover the possibility 
of using materials such as puzzles or of employing syllogisms using 
undefined or manufactured words or phrases. It w’as agreed lliat 
such materials should be used only if they seemed to be the best 
uay to get at certain critical thinking skills. 
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Since there was no rational basis for deciding exactly what per- 
centage of items should be allocated to each cell, the chart was re- 
garded more as a convenient scheme for classifying items to avoid 
imbalance m the test. The aim tvas to fill as many cells as possible, 
although ultimately a decision as to the weighting of the abilities 
was agreed upon. 

These steps involving the definition of the objective of critical 
thinking and the development of a set of specifications for a test 
para e e , rather than preceded, the writing and tryout of items, 
n tact, some of the test specifications already noted were arrived 
at only after use and analysis of the earlier test forms. 


^ * S Natural Values & Non* 

rt., _ . & Ethical Content 

Others Society Physical Stand- Prob- 

Uairerse ards lemi 


Content Axis 

CnnicAi. 

TbINKINQ ABILtTIEa\. 

1. Abdity to dedne a 1 


8 Ability to recognue 

assumptions 

— _ 12 1 S 

■* totmulate 

tetevant hypothese s g ^ 

5. AbJity ,0 draw con. ' 

elusions validly 

L g S 4 

CbMt edi,b,l„g Itc specdicnii<,„ t„r tbc Teat ot Cnlical Thmkmg 

The variety ot°proc'T Critical Thinking 

trying them out, an™revhr' obtaining test materials, 

report in detail. Assienm would take too much time to 

ivrite a certain number ™ade to committee members to 

cells in the chart given abo certain specifications or 

passed on to other facult assignments were in turn 

posed of such items were”^™ Tests and test fragments corn- 
students were asked to writ^'^^ students. For some problems 
of distracters for an objectiv^ ^ response as a basis for selection 
of students’ oiul resDon«#.r! Tape recordings were made 

o objective test items. In other cases 



PERVASIVE OBJECTIVES 1: CRITICAL THINKING 187 

written analyses or defenses by students of their chosen answer 
were collected. All items with information on student reactions 
were extensively reviewed by committee members as well as by 
other faculty members. These procedures were as extensively em- 
ployed in validation as in test development, and will be reported 
in more detail under that heading. Trial forms of tests were devel- 
oped and given extensively to students, so that data on the difficulty 
and the discrimination of items could be computed. From all of 
this effort two forms— A and B— of A Test of Critical Thinking 
were prepared for use with entering freshmen in the fall of 1951. 

The two forms of the Test of Critical Thinking were originally 
developed with the hope that they might be equivalent forms. Form 
A, however, was given the best of the material based on tryouts, 
and Form B was then made up of similar residual material revised 
in an effort to make it equally satisfactory. In tryouts with random 
or matched groups, B was much inferior to A, and its use was dis- 
continued. Form A was used in post-testing freshmen in the spring 
of 1952 and a few sophomores in the spring of 1955. Even Form A 
was not altogether satisfactory, for it was found to be somewhat too 
easy, and it contained a number of nondiscriminating items. Hence, 
still another form, Form G, was prepared. Form G consists of the 
best of the items of all earlier forms. The items retained were those 
which attained the best coefficients of discrimination in the year’s 
testing, but attention was also given to maintaining a proper bal- 
ance in respect to the objectives of the test, and to the selection of 
somewhat more difficult items. Form G was used with entering 
freshmen in tlie fall of 1952 and again in the spring of 1953. Some 
of the results of these testings and associated studies will be pre- 
sented in later sections. 

The specifications for Form G of the test with illustrative items 
will be helpful to those who do not have at hand a copy of the test 
or manual. The illustrative items are similar to those in Form G 
but tiiey are selected from some of the less satisfactory materials 
rejected for the final form of the test, 

1. Ability To Define Problems 

A recently married couple decide that they will give $200 of their 
annual income to cliarity. Their problem is: 

1. \Vliat charity-supported works arc most desenang and in 
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need of the money they can give, and what distribution 
of their $200 best balances these demands? 

2 How shall they deal with those making the appeals, with 
the least time and bother and with the most congenial 
response to their requests? 

3 Which of the appeals that come to them is most worthy? 

4 Whether this amount ($200) is too much or too little for 
the cause they want to aid 

5 Whether concentration of their gift in one agency tvould 
do more ultimate good than distribution of it among sev 
eral users 


2 Ability To Select Pertinent Information 

The head physician at Cowlick College wishes a reasonably ac 
curate estimate of the number of cases of chicken pox treated at 
the infirmary during the past five years, but his time is limited 
A total of 10 000 cases of all kinds of sickness were treated during 
t e period A study of which of the following samples from his 
records would probably be adequate and still conserve his lime’ 

1 Every odd numbered case from the 10,000 

a twentieth case from the 10,000 

group^*^* to represent proportionately each age 

4 The last 400 cases treated 

5 The last 200 cases treated 


3 Ability To Recognize Assumptions 

mem* 47 refer to the following newspaper advertise 

hwtmp^sv^Il ^ planning to install a warm air 

Blake contractor about the advantages of 

saves mnn#« ductwork Many have already found it 

more lono-Z easier for workmen to handle gives 

paintine ^is Tl because it never rusts, never needs 

vents excessive Aluminum's natural insulation pre- 
FHA finanant^ ^ sound is deadened, too Approved for 

his 'vriter makes a number of claims for 

of the stateme^nts 34 thro“"w 7 systems, etc. Mark each 

ments 34 through 47 according to this scale 

th^se'worX states this, although maybe not in just 
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2 The writer does not state this, but he does state 
something which shows that he must have taken it 
for granted 

3 The tvriter does not state this, nor does it have any 
relation to his argument 

4 The writer does not state this, and it would weaken 
his argument if he did state it 

35 Ordinary ducts are harder to handle than aluminum 
ducts 

38 Durability as well as initial expense should be considered 
in buying heating equipment 

4 Abthty To Formulate and Select Relevant Hypotheses 

Two hundred Negro school children and their parents filed suit 
in Atlanta Federal District Court, demanding an end to racial 
segregauon m the schools The suit asked that the separation of 
schools for those of different races be declared unconstitutional 
and be ordered to cease forthwith The Atlanta Constitution, 
the city’s largest newspaper, editonalized, in effect 

EITHER THE SUIT WILL BE WITHDRAWN OR MANY 
WHO HAVE WORKED FOR EQUAL OPPORTUNITIES 
WILL BE ALIENATED 

A group of people, discussing the suit and the editorial, made 
the comments given in Problems 252-257 Mark each of these 
comments according to the following scale 

1 That means just the same thing the editorial said 

2 You disagree with the editorial then 

3 You can’t tel! from what the editonal said whether 
that IS true or not 

252 The suit won't be withdrawn, but not many who worked 
for equal opportunities will be alienated 

253 If die suit is withdrawn, not many who have worked for 
equal opportunities will be alienated 

256 Failure to withdraw this suit will be a matter of alienat 
ing many who have worked for equal opportunities 

5 Abthty To Draw Conclusions Validly and To Judge the Validity 
of Inferences 

In Items 14 through 17 you are to accept as true that all window 
washers are poorly paid, and some window washers have large 
families Mark eadi of the conclusions, H dirough 17 according 
to the follow ing scale 

1. Must be true on the basis of the given statements. 
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2 Might be true on the basis o£ the given statements 

3 Must be false on the basis of the given statements 
14 Some people who have large families are not poorly paid 

5 Some people who are poorly paid have large families. 

7 No people who are poorly paid have large families 

Research Aspects of the Project 
Rehabthty of the Test 

, ^ ts a function of many factors— the test, 
the time allowed, the conditions of administration, and the charac 
pnVi^ T ^ ^ itidividuals tested Students respond rather consist 
toIp^t^ because responding to such tests is a familiar 

ests which are novel in instructions and type of exercise in 
of reliah'iT''^ -u^ random response with consequent lowering 
type of ^ ‘he cnucal thmUng 

in a fewer items can be covered 

reliability 'nterval, and results in a further reduction of 

of two'*houK * t"" ”'“h « rhe length of the test A test 

more reliable ,7°’'^“’® ‘™«-other things being equal-will be 
must be balanced^"he one hour working time Against this 
tee— like the nth. l '"''‘"onee of using the test The commit 

'vouldt shalr ^““‘V-tlec.ded that the use of such a test 

pediency Thw is less sp deliberately sacrificed to ex 

scores were a matte f “"Sbt have been if individual 

demanded The ° major concern or if reliable subscores were 

group changes rather Than™' '‘“‘^1’“'® 
to be used on the test Th "'“fn*'- nnd that a single score was 
forms of the test so th ^ °‘“®‘n^f pfan called for two equivalent 
could be administered ^h * ®‘^^nter reliability were desired both 
length Otherwise altcmi,r’f'" of double 

testing Poor results with FowTr P’’® P”" 

decision to develnn i to its abandonment and the 

One estimate ofThe reTS,T™’p"'°™ 
putmg an adjusted snhf hd ^ Form G was obtained by com 
of the Brst 10 perceS “ ^‘“”Pl<= 

p rcent of the sample from four mstituttons The 
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resulting coefficient was .84 for the 97 cases. Later, Kuder-Richard- 
son formula No. 20 was employed on a 20 percent sample with the 
following results: 

N = 231 Mean = 29.09 S.D. = 7.21 r = .79 

Several institutions reported reliabilities in the range .80-.85 com- 
puted for their own student groups. Such results— while lower than 
might be desired— must be considered satisfactory for the purposes 
for which the test is proposed. Experience indicates that it is some- 
what more difficult to obtain high reliability for tests of this type 
than for content-loaded achievement tests. 

Related to reliability is the performance of the items in terms of 
difficulty .and discrimination. Evidence on both of these points is 
presented in the test manual and need not be reproduced here. 
However, the statistical evidence of this sort on the final Form G 
was excellent, shelving all items making a contribution to the total 
test score. 

Intercorrelations of the Critical Thinking Abilities 

Related to the reliability and to the decision to avoid part scores 
is evidence on the interrelation of the groups of items classified 
under the five major abilities. One such set of intercorrelations is 
given in Table 38. This is based on only one institution but indi- 
cates the general order of magnitude of the correlations as verified 
for other colleges. 

Form C, made up of all items from Forms A and B and other 


TABLE 88 

1nt£bcoii6z:i>atzov or Kbtutus Tested ov Test or CbiticjUv TniJfEixo 
freshmen and sophomores) 


Pars 

Ko or 

1 Ima 

1 

1 Coasiunox Cocmcicm 

DefiaiUon 




Drawin* 

CoodufioDi 

Definitions.... . . 

6 

X 





Selecting Information . 

12 

.15 

z 




Kecognizmg Assumptions 

14 

.10 

.23 

z 



Seleetmg Hi-potheses 

10 

.OS 

.20 

.13 

z 


Drawine Conclusion-s 

15 

.23 

.23 

.42 

.26 

z 

Total 

S7 

.40 

.SI 

.67 

.64 

.73 
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Items employed in the tryout forms, was developed solely for re 
search purposes. This form was divided into four sections dealing 
respectively with the abilities 2. 3, 4, and 5. Information based 
upon these longer tests of the major abilities incorporated in Form 
suggests that perfectly reliable tests of these abilities would inter- 
correlate to the extent of about .65 to .85; such intercorrelations 
^int to t e fact that the abilities are somewhat distinctive and 
^ T subtests might have some diagnostic significance not 
available in the present short form (see Table 39). 


TABLE S9 

taconneunos or Scones on Tee PnnTS or iBa T«t or 
CrRmcAL TniNKraa, Form C 


C-l 

C-2 

C-3 


EtTnuTE 
o» Rtu 
ABimr 


72 

76 


CoSBELAnOM CoimcIENT 


.57 

.63 


Validity of the Test 

Thinking'" Commfttee'ind problem for the Critical 
acteristic of a test but t!"*’ intrinsic char- 

situation in which tho , '^''."“'eristic jointly of the test and the 
dents to whom little e srhuols wherein or for stu- 

critical thinkino- sV ii ** provided in the development of 

validity a ” thinking can tile much 

It is much as though a Lt educational objective, 

the development of ? tnathematics were employed to assess 

the matheLt™ mst fo'“ ' reliable 

that situation. ^ purposes, it would not be valid in 

The natural wav to rh i. 

with some direct criter* validity is to compare that test 

quate means available T” ° ability tested. If there were ade- 
judgments were thorouffhl critical thinking or if teacher 

need to construct a te t ^f^^**^*^^®*^* there would have been no 
avoid the responsibilitV of ^'Huments are not presented to 

rather, they are meant / i. sonie evidence of validity; 

y meant to show that the task is not a simple one. As 
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with other of the tests of the Study, our case for validity, then, must 
rest on the cumulative significance of a large number of both posi- 
tive and non-negative findings and on the steps taken in writing 
the tests in the first place. Unfortunately, most of the evidence 
discussed in this and following sections is based on Form A rather 
than on Form G, but there is evidence which suggests that Form 
G (which is, after all, only a slight revision of Form A) is more 
satisfactory than Form A. 

The first measures to ensure validity were those involved in the 
process of the development of the tests. The judgment of the 
faculty members that the tasks involved in the items were mean- 
ingful and educationally significant is not to be taken lightly, but 
no one could be more skeptical of this judgment than the com- 
mittee itself. Hence this judgment was buttressed by: (1) Record- 
ings were obtained where students verbalized (with prodding, if 
necessary) their responses to critical thinking questions in front of 
a tape recorder. Their thought process was then compared with the 
one hypothesized, and flaws in the items were also uncovered. (2) 
Test items were restructured so that the stem only could be pre- 
sented to students for an essay response. This provided a check on 
the thinking involved and suggested responses suitable for the 
objective form. (3) Test items with ansivers were presented with 
the instruction that the answer should be selected and then a para- 
graph written to justify or defend the choice. 

These checks ensured, for most of the retained and revised 
items, that correct thinking would result in selection of the correct 
answer, and that the wrong answers corresponded to incorrect 
thinking or to guessing. This evidence, combined with the judg- 
ments of the committee, also helped in selecting items which can 
and do evoke the mental processes originally accepted as descrip- 
tive of some aspects of critical thinking. 

Another approach to the investigation of validity involved the 
use of two criterion groups made up of students classified by teach- 
ers, residence hall counselors, or other students as evidencing 
critical thinking or lack of it. Several such studies were carried out 
independently at many institutions. 

Dr. Lily Detchen requested Uventy-two instructors at the Penn- 
sylvania College for Women to rate on critical thinking all the 
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seniors well known to them. The ability was discussed with the in- 
structors to the point where they understood the qualities involved, 
and they then rated students as superior, average, or inferior. An 
item analysis (of Form A) was then made on the basis of the teacher 
ratings and compared with the item analysis based on the total test 
score, with the results shown in Table 40. 


TABLE 40 

Item Analtbis op A Test op Critical THiNKma 


Analysis 

Found acceptable by teacher rating but not by test score 
criterion ’ 

und acceptable by test score criterion but not by teacher 
rating ^ 

Found acceptable by bath'mVtiiodV 

Found unacceptable by both methods i 


No of 
Items 

4 

9 ' 
34 
10 


Total number of items in test 


57 


comsnnnd items were found to make discriminations 

ratinerwerl'? teachers’ judgments. When the teacher 

Table 41 were ”oun"^d- 


table 41 

Com..n„o» ue 

OF Critical Thinxino 


PEKrOBMANci OH 

TBS Tsst 

[ Tucbcb IUttno 

Total No or 
Students 

Oeatly | 

supenor | 

C'early 

lafenor 

Above median 

Uelow median. . 

19 

1 

20 


10 

12 

— ^ 

21 

11 

S2 


College on a group'' '=*"=“i''e data at Stephei 

Thinking. All were ^ Test of Critic 

advisers, residence hall Psychology instructor 

viding ratings of critical .l"'! ■ ““dents cooperated in pr 
cal thinking utilizing the forms shown i 
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Figures 7 and 8 Although not all these ratings could be obtained 
for all cases, the volume of evidence svas still impressive The re- 
sults are shoivn in Table 42. 


TABLE 42 

CoSBZIATIO't BETWEEN StODENTS ON RaTIHGS OF CrITICAI. ThIKKINO AND 
THEm ScoRE3 ON A Test of CbiticaIi Thikkiko 


BaYCIGS ST 


1 RATtNOS * j 

[ Tist Scoaa* 

COBSEU 

WOT 


M«An 

SD 

Meca 

SJ> 1 

psychology teacher 
Adviser 

S23 

3 22 

1 29 

118 40 

31 54 

66 

336 

3 83 

1 07 

118 78 

31 35 

50 

Mali counselor ^ 

360 

3 40 

1 00 

119 SO 

21 12 

42 

Selr 

344 1 

3 od 

72 

119 90 

21 00 

50 

Peers (at least three ' 
raters) ' 

1 

166 

3 59 

67 

120 00 

31 08 

62 


* Composte o! acreral tomu of tbe ie«t. 


Developing a good test is a very complicated business, and so vve are 
asking your further help in detcrmmiDg how good the experimental 
Test of Critical Thinking is 

1 Your name - _ _ 

2 How well do you think you did? Don't be modest and don’t brag 

5— as well as the highest fifth of all students 
Circle 4^as well as the high fifth 

one 3— as well as the middle fifth 

number 2— only as well as the low fifth 

1— only as well as (he lowest fifth 

3 How well do you think others )ou know did? 

3 1 List below the names of students taking psychology whom you 
know well enough to make a guess as to how well they did 
3 2 Make your guess by comparing them with yourself, and then 
deciding on and wriung in the chosen number from 2 above 


A( ). 
B ( ). 
C ( ). 
D ( ). 
R( )- 
F ( ). 
G ( ). 


Fic 7 —Form for recording student judgment of the Test of Critical Thinking 


















HOW ABOUT A GOOD SCOUT DEED? 

Stephens is a participant m the Cooperative Study of E\aluation in 
General Education One instrument being developed is a Test of Critical 
Thinking ^ 

This CT test IS designed to measure how skillful the student is in 

a) defining and analyzing problems 

6) recognizing unstated assumptions in an argument- 
c) determining the relevancy of data 
<i) drawing valid conclusions or inferences from data given 
e) formulating and testing hypotheses 

The Items of the test were chosen so that as nearly as possible 

answer would depend upon seeing and judg 

b) th^ Ite^ ® background of knowledge 

ifchalkngmg'* situation to make 

c) the Item would be emotionally neutral 

Slow! ould®dfon^he tesf Y " 

determine how good the test is ““P'”"”" «'» definitely help to 
the twdS m dm^'th“hmT^f’’th 'k°‘ T 

not how able he i< ^ thinking described above, 

about s™me matyrT°'*°"* with his doing straight thinking 

following ^ ^ Jttdgraents use the appropriate number from the 

1 S J "■e low fifth 
^ 1-should score m the lowest fifth 

to have even a vaeue^rZnr***^”*^ whom you have observed sufficiently 
M«k X for dice Xr”" n “ ‘'■■"k critically 

1 ( ^ insufficient observation 

2 ( ) 6 ( ) 

3 ( ) M ) 

t < > IH T 8 ( ) 

5 < ) H Z 8 ( ) . 

■ • •“ ( ) 

_____ ’•‘y^holoe, D,p„rtmM 

enum, Thinking 

196 



Test Score 
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The mean ratings in Table 42 compare reasonably well with the 
test scores. It is worth noting that the psychology teachers who 
knew the test reasonably well and the students ivho had taken the 
test both made ratings which correlated more highly than those of 
other groups. Students tended to rate themselves lower than did 
the staff or peers. This is easily seen in Figure 9 tvherein are plot- 
ted the mean ratings and test scores for the five different ratings. 



Fic. 9.— Rclaiion between ratings and Critical Thinking Test Scores. 

At Kansas State College four instructors provided critical think- 
ing ratings for seven classes which were compared with the test 
scores. The reader for one of the instructors also provided ratings. 
The pooled ratings compared with the test scores are shown in 
Table 43. 

Chi-square, a measure of association, is found to be 23.67, svhich 
is well beyond the .01 level (chi-square equals 13.28 for 4 degrees 
of freedom). Analysis of the separate ratings, however, sliows sig- 
nificant association at or beyond the .01 level in only two classes. 

Another bit of evidence relevant to the issue of validity is found 
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TABLE 48 

TEAcmais’ Bating on Critical Thinking 


CnmcAL TffiNiiso 

Tkacbeb Ratixq 

No. OF 


Low 

Medium 

High 


Low . 

15 

16 

4 

85 

Medium 

14 

SO 

IS 

66 

High . .. 

1 

21 

17 

89 

No. of items 

SO 

76 

S4 

140 


in the reactions of students to the test. Such reactions were taken 
in a number of colleges and were found to be generally favorable. 
Of one group of 207 students: 

120 reported enjoying the test; 

54 reported indifference or mingled reactions; 

33 reported dislike of the test- 

Reasons commonly given by those who enjoyed it used such 
phrases as: “challenging,” "increases power to evaluate facts," 
good mental exercise," "had to think for a change." Those who 
disliked the test reported in such phrases as: "ambiguous," “too 
ong and monotonous,” “problems over my head," “felt as if I 
1 n t know anything. Clearly, all students found it to be a test 
requiring a somewhat different type of mental activity than those 
to which they were accustomed. 

Intercorrelations with Other Tests 


The mtercorrelation of A Test of Critical Thinking with other 
u some additional basis for understanding the test, 

c ata axe also rather tenuous evidence of validity which can be 
ih' negative rather than positive. For example, if critical 

IS not identical with intelligence, then a test of critical 
correlate very highly with an academic apti- 

; . other hand, a low correlation with 

Tf thil^type 

Tcpotted in Table 44 are selected from many 
the colleges. These were chosen either because of 

evidpnrp v ^ ^ey represented unique 

hprp ^ VK gioups. Correlations reported 

« e various tests of intelligence and aptitude range 
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from .38 to .71. Although there are exceptions, the Test of Critical 
Thinking, as expected, shows somewhat higher correlations with 
verbally loaded over quantitatively loaded tests. 

TABLE 44 


CORSEIATIONS OP A TesT OF CriTICAI. ThiNMNO WITH IntELUQENCE 
OR Academic ApirroDE Tests 


Name of Teat 

Gronp 

N 

r 

ACE Psychological Examination 

Total Score 

Freshmen 

SS6 

.38 

Total Score 

JVeshmen 

SOO 

.69 

Total Score 

Freshman Women 

20S 

.40 

Lbguislic Score 

Freshman Men 

339 

.42 

Linguistic Score 

Freshman Women 

SOS 

.55 

Ohio State Psychological Examination 

Freshmen 

165 

.65 

OEEB Scholastic Aptitude Test 

I'Yeshman Men 

SIS 

.60 

Yale Aptitude, Verbal Keasoning 

Freshmen 

166 

.52 

Yale Aptitude, Quantitative Ileasoning 

Otis Self-Administering Tests of Mental Ability. 

Freshmen 

166 

.52 

Freshman vVomen 

65 

.71 


TABLE 45 

Correlation op Critical Thinkino Test tvith Variods Achievement Tests 


Name of Tert 

Source of PaU 

Group 

S 


Cooperative General Culture 
(History of Social Study) . . . 

Antioch College 

Freshmen 

165 

.50 

Nelson-Denny Reading, Form A 

Vocabulary 

Paragraph Comprehension. . 
Total . . 

Pennsylvania College 
for Women 

Freshman women 

r r 

0 r 

150 

250 

150 

.37 

.50 

.56 

MSC Readlnt' 

MSG Reading Test 

MSC Reading Test 

Michigan Stale College 

Freshman men 
Freshman women 
Grad, students 

164 

134 

10 

.49 

.53 

.80 

GED College Level Social 
Studies. Form R 

Geo College I.evel 

Correctness and Effective- 
ness of Expression, Form B . . 

Pennsylvania Collie 
for Women 

Freshman women 

r r 

150 

150 

.56 

.47 

Owperative Reading (C2) 
Vocabularv. . 

Speed. 

Level 

Total 

Stephens College 

College Women 

» » 

» 0 

Cil 

Oil 

C41 

041 

.57 

.60 

.55 

.65 

Iowa High S<diooI Content 
Enelwh. . . . 

Malheraatirs. 

Science. . . 

nislorj" and Social Studies. . 
Total 

Pennsylvania College 
for Women 

Scholarship appli- 
cants (85% of 
them above me- 
dian on Iowa 
High School 

Content Total 
^re) 

80 

80 

80 

60 

80 

.39 

.63 

.33 

.47 

.67 
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There being no planned program for relating the Test of Criti- 
cal Thinking to subject-matter achievement tests, the correlations 
reported in Table 45 are the result of local investigations. Except 
for the correlation of .68 with the Mathematics section of the Iowa 
High School Content Examination, the correlations are not sur- 
prising. Despite the generally high level of the scholarship appli- 
cants at the Pennsylvania College for Women, the Test of Critical 
Tliinking was found to have a good range and was used along ^vith 
the Iowa High School Content Examination scores in making de- 
cisions about scholarships. 


TABLE 46 

CORREL.<TIOV or THE Te9T OF CsmCAL TnTNXmO WITH 
CoCRSE EtVillNATlONS AND Gr.ADBS 


CoQTfe er Ex*minAlioti 

Institutioii 

Group 

N 


Rr^.jfsr aTerage/Criitcal 
ThmkjBg pre-lest 

First-yfiir arerage/Critical 
Thinking poat-test 

Four.ycar point average 
lU^tory of Western Civdiia. 
tion 

Biology... 

Freshman English * . ' 
Human Development* . , 

Pennsylvania College 
for Women 

Freahxaan women 

Freshman women 
Senior women 

Freshman women 
Freshman women 
Freshman women 
Freshman women 

144 

144 

101 

50 

50 

50 

50 

.58 

.69 

.St 

.52 

.60 

.49 

.70 

Grade-point average. 

Stephens College 

College women 

m 

.49 

Grade-point aserage in gen- 
eral eduntion courses 
brade-poinl average m all 
other courses 

Michigan Slate College 

Juniors and seniors 

Juniors and seniors 

67 

67 

.62 

.15 

Practical logic comprehensive 
ciammation. . , 

University of Florida 

Freshmen 

S83 

.50 


amHwn in noun DcTrkpiMet iiK)q<i«| (ome rTpentocntnl cnbcn] thmkiDS item*. 


Tlie data from Table 46, like those from Table 45, were re- 
jwrtcd as phases of voluntar>- local studies. Correlations between 

c ® *^^‘*^^**^0 scores and grades commonly equaled 

Tpf ' p I of academic aptitude tests such as the 

' S)c 10 ogical Examination with grades. The surprising dif- 
cr^cc m correlations with general education course grades and 
wuh other courses found at Michigan Slate College is thought to 
r«ult from tlie cmpltasis on critical thinking in the general educa- 
tion courses- lis emphasis is reflected in examinations which re- 
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quire this ability in considerable measure and which determine 50 
percent of the grade. 

In general, the correlations shotvn in the preceding tables are 
in the expected range. White correlating reasonably well with tests 
of intellectual ability, there is no reason to classify the test as a 
test of intelligence. The correlation is almost equally good with 

TABLE 47 


Typical CoRHEL.mova or the Test op Critical TmKKcia iraii 
Other Tests op Critical TniNxmo 


Te«t 

tcstitulion 

Group 

N 


Science Reasoning and Under- 
standing, Form A 

Pennsylvania College 

for Women 

Antioch College, . 

Freshman women 
Freshmen 

150 

40 

.89 

.50 

Critical Analysis in Reading 
and Writing 

Pennsylvania College 
for Women. . 
Rficbigan State College 
Michigan State College 

Freshman women 
Freshman women 
Freshman men 

150 

.53 

.51 

.82 

Critical Thinking in Social 
Science 

Pennsylvania College 

for Women 

Antlodi College 
University of Horida 

Freshman women 
Freshmen 

Freshmen 

150 

85 

SOO 

.09 

.08 

.07 

Critical Judgment in the Hu- 
manities (Literature) 

Western Washington 

College of Education 


98 

.88 

Watson-Glaser Critical Think- 
ing Appraisal 

Michigan State College 
Michigan State College 

Freshman women 
Freshman men 

104 

134 

.01 

.53 


various achievement tests and with grades. In fact, as already re- 
marked, most of the colleges computing such correlations found 
the Test of Critical Thinking slightly higher in correlation with 
grades than their usual academic aptitude test. Higher correlations 
were found with grades in certain courses where a priori considera- 
tions indicated greater emphasis on critical thinking. One of the 
most outstanding of these is the correlation reported from one col- 
lege of .62 with general education course grades against .15 for 
otlier courses. On the other hand, a course in practical logic sup- 
posedly specifically aimed at this objective yielded a correlation 
of .59, no higher than that found for otfier courses or combinations 
of courses. Correlations with the Test of Critical Thinking in the 
Social Sciences ran rather consistently above .60, showing that 
these two have much in common despite difference in the content 
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and problems utilized. It was the judgment of all directly con- 
cerned that the results were consistent with expectation. 


Measurement of Change in Critical Thinking Test Scores 
A major concern throughout the work of the Critical Thinking 
Committee was the accumulation of evidence on the extent to 
which critical thinking ability is or can be improved by education. 
Still more specific is the question of the differences in development 
induced by varying patterns of general education courses or by 
general education courses as against more traditional departmental 
offerings. The issue of the relative effectiveness of a concentrated 
course in logic or critical thinking as against the more diffuse em- 
phases in a number of courses is a pertinent one. Although some- 
thing more can and will be said about some of these matters in 
chapter 10, it may as well be stated that no answers are available 
here for such questions as those just posed. A variety of types of 
^idence, however, will be presented indicating that A Test of 
Critical Thinking does show evidence of differences in student 
groups and that this evidence bears some relation to the educa- 
tional experience. 

^ For such comparisons, pre- and post-test data from a number of 
groups of students matched in terms of 
a 1 ity are available. In general, students do make higher scores 
a ter one and two years of college, and student groups of successive 
>ears s ow continued but decreasing increments in tested ability. 

*uch data, two prior questions need to be con- 
r S‘*^ce Form B was discontinued, all repetitions uti- 

Vnn ^ ^ form. Therefore, there is some concern attached to 
wit \ ^ effect. Investigation of this in two institutions 

wherein the same form (Form A) was repeated at intervals of a few 
months (embracing very little class work, however, 
difFrrfn ^ r ^ *"**^*^^”*"5 Christmas holidays) revealed mean 
h n point. The practice effect, then, 

time when ^ >'gi>ificant for longer intervals of 

eai!I' ""nt to which differences in 
gams might be simply due to differences in the original groups. 
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TABLE 48 


Pre-Test Means and Standard Deviations on A Test ov 
Critical Thinkino Form A 


Institution 

N 

ol 

Scores 

&fean 

SJ) 

A 

836 

48-11 

80 24 

7 01 

B 

124 

46-15 

81 28 

7 02 

C 

271 

48- 5 

26 96 

7 44 

D 

250 

46- 8 

27 61 

7 41 


Four of the colleges making most extensive use of the test had the 
pre test results shown in Table 48 The means were found to differ 
significantly, but the ACE Psychological Examination means were 
likewise found to differ significantly Adjusted for differences in 
ability, the means for A Test of Critical Thinking were as follows 


Institution A 29 51 

B 28 84 

C 29 48 

D 28 76 


Differences are no longer significant This may be taken as an 
indication that, allowing for ability, the students entering vanous 
colleges (the four above were very different in their programs and 
in the type of students attracted) do not differ in critical thinking 
ability 

Table 49 shoivs combined data for a number of colleges on the 
pre test and post test results over the freshman year The difference 
between means computed from Table 49 might be inflated because 
of selection involved in the post test Still another factor involved 
in dealing with gams is the negative correlation of gains with pre- 
test These negative correlations are generally small, the largest in 
absolute value being 31 over a period of four months Over longer 


TABLE 49 

Data Derived from Rub- and Post Testino Freshmen, 
Tist or Critical Trinelvo Form A 




UcsNScoacs 


No or 
StvoDm 

No or 
COLLMCa 

Tmt j 

lowest 
Iteported by 
* CoUege 

ntfliest 
Reported by 
■ CoOece 

AO 

Slodeats 


20 90 

37 85 

3<» 15 

2,171 

12 

Po*t test 1 

S3 00 

42 09 

36 03 

743 

5 
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intervals the correlations were in the neighborhood of —.15. Since 
this phenomenon is a rather familiar one and not unique to this 
test, further comments and possible explanations will be reserved 
until later. The implication is, however, that gains are meaningful 
only as they are related to pre-test status. A school or individual 
with high initial performance cannot be expected to do as well in 
gains over one or two years as a school or individual with low 
initial performance. 

In order to arrive at Table 50, all pre-test scores in colleges con- 
tributing 45 or more pre- and post-test scores for the same students 

TABLE 50 

Guns or SnroEjrta o'* A Test or Critical Tuineino, Form A. 

CiAsairtED AccoRomo to PnE-TEaT Stai^oino 
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made for lower pre test groups m other colleges From Table 50 
several significant facts emerge (1) Students in different colleges 
vary considerably in gams made in critical thinking (2) Because 
of the differential gams due to pre test status, total group gains are 
not the best index of program effectiveness For example, College 
No 1 achieves excellent results for the four lower groups but has 
so many students m the top group that the total group gam is rela 
lively small (3) The colleges showing the larger gams have quite 
different educational programs so that no simple explanations are 
available (4) The college with the largest gam places consider- 
able emphasis on a comprehensive examination program which 
heavily emphasizes thought type items rather than simple recall 

The numbers m Table 50 make risky any conclusions, however 
tentative Nevertheless, the general picture presented is confirmed 
by more extensive data involving students unmatched from pre to 
post test The test does point to the differences m the growth of 
students in critical thinking and is suggestive, therefore, of the 
generalization that the set of abilities included m the test can be 
developed if sufficient thought is given to providing an appropriate 
educational experience 

Table 51 shoivs the gams when students are classified both by 
ability and pre test status The tendency is for students whose 
ability level is relatively better than iheir pre test level to make tlie 
larger gains 

Numerous small mvestigauons were made comparing the critical 
TABLE si 

Gains or Students on Test or CnmcAi, TniNxiNa Form A, Post TtsT, 
Ci.xssinEO According to Quantile Rank on the ACE Pstcuological 
Examination and the Ifer or Critical TiiLNKiNa, 

Form A Pre Test 


r. ace 

P»TCBou>ateAi. 

1 QoAkmx IUkk Tear or CsmciL TBPfKca Foaw A 

Total 

rint (lo 

w) 

1 Second 1 

\ Tbifd 

1 Fourtb 


Gta 

X ' 

Ctin 1 

JU 

G«>b 

N 

IQQI 


Gtia 

N 


11 sa 

36 

7 73 

22 

5 40 

IS 

1 05 


1 D 15 

73 

Second 

11 10 

la , 

5 07 

18 

6 20 

20 


10 

1 5 80 

73 

iliifd 

13 S5 

li 

D 81 

21 

2 7C 

17 

1 3 20 

23 

0 73 

73 

lourlh 

16 00 


0 53 

12 

fi 00 

21 

2 85 

SI 

C CS 

”, 

'i'olal 

1* S3 

m 

8 12 


6 48 


2 52 

73 

7 n 

iOi 
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thinking scores o£ matched groups and the gains made by matched 
groups In these cases the matching was done in terms of ability 
Out of these such generalizations as the following emerge (1) 
Greatest gams are made in the freshman year, a typical picture 
being a four point gam the first year (2) Older freshmen and up- 
perclassmen show less progress than freshmen of average or below 
average age 


Relation to Instruction and Courses 

There was no direct relationship between the work of the Cnti 
cal Thinking Committee and any particular type of course It is 
not to be expected therefore, that many direct implications can be 
drawn about modifications m classroom procedure In fact, the task 
defined by the committee involved a concept of critical thinking 
as a set of problem solving abilities transcending particular courses 
and course content 

Such evidence— and it was unsystematic in nature— as became 
available gave no advantage to those institutions having a special 
course dealing with critical thinking In one particular investiga 
tion pre and post testing m a course in logic-done by Dr Charles 
F Virtue University of Maine— no marked improvement was 
noted on the Test of Critical Thinking Dr Virtue remarked that 
the objectives of such a course probably do not coincide with those 
of the test Logic— in his analysis— aims at development of a critical 
appraisal of thinking but not necessarily developing better think 

Likewise it is unclear whether general education course expert 
ence has resulted in any greater development of critical thinking 
in students than results from other courses Consistently the largest 
gains were recorded in the freshman year and subjective appraisal 
of the three or four programs showing the largest gam would indi 
cate t at these involve some of the better organized general educa 
tion experiences for the freshman year Pre and post test results on 
a one basis for advanced students reveal gams much smaller 
than those of freshmen What the results might be over a year for 
in iji ua s not enrolled m college or in any educational program 
we do not know Several groups that were contacted on this point 
lacked interest in cooperating 
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Summary 

The Test of Critical Thinking provides a way of studying the 
growth in certain aspects of critical thinking of single groups, or 
the relative performance of several classes. Data presently available 
provide some basis for noting whether gains are as great as have 
been found elsewhere. The nature of the program which will pro- 
duce greater gain remains unspecified, but it may be significant 
that those colleges recording the largest gains were ones with 
courses specifically organized for general education purposes and 
with definite requirements for all students involving the comple- 
tion of all or a major portion of these courses. 

The marked interest shown by students taking the test and their 
reactions to it suggest that the test does require a type of mental 
activity different from that found in many courses. Judgments by 
residence hall personnel, instructors, and peers suggest that the 
ability measured by the test has some carry-over to behavior. There 
is also reason to believe that it is not synonymous with intelligence. 
Experiences of some of the other committees concerned with criti- 
cal thinking suggest that more appropriate course experience can 
be developed to make a heavier contribution to these abilities. 
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Intercollege Committee on Attitudes, Values, and Personal Ad 
]ustment Gladys Bellinger, Kansas Slate College of Agriculture 
and Applied Science, David Burkhart, Boston University, Ruth 
Churchill Antioch College, William Damson, Stephens Col 
lege, Maurice Freehill, Western Washington College of Educa 
tion Leo A Haak, Michigan State College, Darrell Holmes, 
Muskingum College, Arno Luker, Colorado State College of Edu 
ration J L McCreicht, Muskingum College, Milo Milanovich, 
Muskingum College Samuel Neel, Florida State University, 
C Robert Pace, Syracuse Universit), Mary Katherine Smith, 
Muskingum College, George Stern, University of Chicago, 
Cornelia Wiluams. University of Minnesota 


T he nineteenth century emphasis on positivism on scien 
ti c rigor, and on knowing and teaching historical facts 
eseigenthch gewesen war led teachers in early twentieth 
"^ci’ican colleges and universities to attempt the impos 
fn tL ^ affect the intellect of students without reference 

cognitive being Students in colleges and universities 
whatever knowledge to which they ivere 
concomitant, the acquisition of skill m 
3 b>hty was accepted without question Beyond the ac 
11 ° development of intellectual adroit 

ne« college teaching assumed no responsibilities 

fhp r,. , *^***^iHg of John Dewey, Thorndike, 

^ sc oo of psychology, and progressive education has 
e in a rejection of such a restricted concept of education 
u ents are now presumed to enter school complete with bodies, 
208 
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good and bad habits, attitudes, and principles of behavior, all in- 
teracting to form the entity which is to be educated. It is true that 
this concern for all aspects of the human organism has been more 
manifest in the kindergarten and elementary schools than in higher 
education. The unbracketing of chairs for the convenience of 
wiggle-worm children, the providing of milk at recess to stoke their 
perpetually mobile engines, and the provision of subjects to appeal 
to their ever-changing interests, are all exemplary. High schools 
and colleges have been much slower in realizing that the organisms 
inhabiting their classrooms are not bodiless intellects to be molded 
in the images of their mentors. 

Of recent years, however, higher education, particularly at the 
policy-making level, has verbalized a concern for some of the non- 
cognitive aspects of learning. The Report of the President s Com- 
mission on Higher Education lists eleven goals or objectives for 
a general education designed to meet the needs of an ever-expand- 
ing number of American youth. Each one of these major goals 
recognizes to some degree the importance of an affective domain 
among educational objectives. Thus, students are expected to 
develop a code of behavior based on ethical principles. They are to 
participate as responsible citizens. They should recognize personal 
responsibility for fostering international understanding. In addi- 
tion to learning facts about their physical environment, they 
should appreciate the implications of scientific discovery for human 
welfare. They should attain a satisfactory emotional and social ad- 
justment. They are to enjoy literature, art, and music, and should 
acquire attitudes basic to a satisfying family life. Their selection of 
a vocation should be socially useful and personally satisfying, and 
should allow an individual to make full use oi his interests. Even 
the most intellectually oriented of these objectives have their 
affective concomitant. Thus, under the major goal of acquiring 
and using the skills and habits involved in critical and constructive 
thinking, students ate expected to develop a stimulating intellec- 
tual curiosity.^ 

Even more significant than such statements of objectives ave 
been the concrete developments of college administration and cur- 


'Hish„ Education tor American Dcmocrac,. Vol. I-Ectabiulnng ^ 
iagtem: Government PrinUng Office; and New York; Harper 8: Broi., 1917). pp. SO-S7. 
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ricular practice emphasizing the same concern The emergence of 
university supported groups of trained counselors on college 
campuses to advise with students about the many problems, intel 
lectual or emotional, with which students are faced, the provision 
of well staffed health centers, the increased emphasis on dormitory 
living for male as well as for female students, and the evolution of 
courses variously labeled Effective Living, Human Growth and 
Development, Human Understanding, or Human Relationships, 
all attest to the growing importance attached to education for the 
whole individual 

In view of such interest and concern, it was logical for the Co- 
operative Study of Evaluation in General Education to demon 


strate an active interest in affective as well as m cognitive areas 
From Its inception there was no doubt that one of the major 
operating groups of the Study should concern itself with some non 
intellectual outcomes of education The vehicle selected for this 


purpose was an intercollege committee dealing with how general 
education affected attitudes, values, and personal adjustment 
le title of this committee, which was composed of counselors, 
professional evaluators, and teachers m courses dealing with 
luman relations, furnished such a broad field of possible opera 
ions t at considerable self restriction had to be imposed before the 
^oup could make its efforts effective The first modification con 
the concept of personal adjustment from active 
consideration by the committee There was never any thought that 
personal adjustment was unimportant There was the belief, how 
vidiiii P^t^^na adjustment varied so from individual to indi 
not mnl P* place, that any study 

unaltle tn ^ psychotherapy would be 

concentrated T" ntcaningful evidence Thus, the committee 
Ft en d “‘a™"™ ™ “"d Values 

issues tn personal adjustment left many further 

come The lem*'" many technical difficulties to be over 

mat ranee ante I, many and varied meanings which 

as tie'ttmild^'^ an etliical principle of doing unto others 

hein^oror a “> a pronounced feeling of 

teclinical wo something The definitions of attitudes winch 

teclinical worUrs tn this field have advanced furnish almost as 
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much variety. Most, however, involve elements reflected in the 
definition, “An attitude is an emotionalized tendency to act for or 
against something.” This underscores the hypothetical aspect; that 
is, an attitude is like intelligence— one will never see it, one must 
only infer it from certain overt behavior. It further links attitudes 
clearly to the affective or emotional domain. Some individuals 
contend that attitudes which are not intellectually informed are 
unimportant. The definition advanced rejects this contention. An 
attitude, that is, a feeling for or against something, may be of 
crucial importance and still be completely uninformed. It may be 
based solely on an emotiona\ feeling. This conception of attitudes, 
accepted by the Attitudes Committee, carried as concomitants 
many vexatious problems. Clearly there could be as many attitudes 
as there are objects or orders of objects toward which individuals 
could be for or against. Even in the restricted area of general edu- 
cation there are literally thousands of objects which could reason- 
ably be of concern to a committee investigating in this area. Atti- 
tudes toward general education, toward specific courses, toward 
teachers, toward certain principles, toward theories, even toward 
historical figures, were possible, and each of these would be of 
some interest to certain schools, teachers, or students. Obviously, 
attitudes toward a variety of discrete objects could not be appraised 
by a relatively small committee operating for a short period of 
time. 

The committee developed an hierarchical conception of atti- 
tudes which hypothesized that attitudes toward discrete objects 
—for example, particular persons, particular books, or specific 
events— condition attitudes toward more generic forms of the same 
objects. Thus, attitudes toward Das Kapital, Mein Kampf, and 
Mother Goose rhymes, condition attitudes toward books. Attitudes 
toward Joe, the Negro janitor, condition attitudes toward Negroes. 
Attitudes toward these general categories condition one’s feeling 
of for or against even higher levels of abstraction. Thus, attitudes 
toward Negroes, Jews, Chinese, and white American Protestants 
condition one’s feeling toward human beings generally. As atti- 
tudes increase in their degree of abstraction, they can be conceived 
of as ethical principles or values which interact with other ethical 
principles of values. This entire hierarchy of attitudes rests upon 
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and in turn affects the biological and psychic aspects of personal 
ity Thus, the committee hypothesized that attitudes toward van 
ous specific objects were conditioned by certain personality config 
urations which resulted from a particular course of development of 
some individuals physical, mental, and emotional traits This con 
ception allowed selection of areas of investigation at a fairly high 
level of abstraction Information about these abstract attitudes 
would then permit inferences to be made concerning more specific 
manifestations of the same attitudes 
Before making final selection of the precise nature of its inquiry, 
the committee developed a series of four assumptions of general 
working principles which were to govern its activities 


1 ^leasurement is a process of making inferences and judgments from 
serve samples of behavior An attitude test is a collection of be- 
samples The validity of measur 
pjgjj epend upon the adequacy with which behavior is sam 

consmen^r^^^ several dimensions-direction, range, intensity, and 
sene op of measurement must be designed to ob 

dude all different situations to m 

ECneral^dilr-fN interested m attitudes as outcomes of 

which the rnm ^ wholc Whenever possible, attitudes with 
ment of concerns itself should be those to the develop- 

•00 ™portant to caution itseU and others against 

tutement "•'“"■"S ■nstmments o£ the verbal 

are also easiesffni- ^ i. are the easiest to construct but 

•hat such instrumtm*' •“ falsity It has been found 
actual behavior ^ discouragingly low correlation with 


then Irrf to^sel<v-V?b'*''^* criteria for selection, the committee 
find or develon a ' *?'"••••<• attitudes with which to work and to 
•o .he setomn?r°’’' c As a prehm.nary 

basis for discussion tuth local '''' 

students Out of such A ‘■“‘‘‘“bonal staff members and with 
Out of such discussions tt was hoped that interest toivard 
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some particular complex o£ these would crystallize to guide the 
committee in its further work. It will be noted that the rubrics 
under which these statements were subsumed corresponded almost 
exactly to the rubrics of behavior developed by the Critical Think- 
ing Committee. 


ATTITUDES OF CONCERN TO GENERAL EDUCATION 

A. Attitudes toivard self such as: 

1. Conscious personal development- 

2. Respect for others coupled with humility for self-seeing the 
individual in the perspective of the group. 

3. Understanding and acceptance of a bi-sexual world. 

4. Willingness to look honestly at one’s self. 

5. 'Willingness to gain self-knowledge through inventories and 
other objective sources of information. 

6. Moderate elation or disappointment in terras of the long view. 

7. Favoring goal-setting consistent with ability to achieve. 

8. “Emotional maturity.” 

9. Giving and receiving affection freely. 

10. Self evaluation (inadequacy vf. overestimation). 

11. Self-asseruon (demanding vs. complete yielding to demands of 
others). 

12. Aggressions (hostile vs. guilt-submissive). 

13. Undemanded creative effort— joy in one’s work. 

14. Appraisal of achievement in terms of one’s possibilities rather 
than in terms of the performance of competitors. 

15. Focus on areas of greatest interest— willingness to let some 
things go. 

16. Recognition that daily life is something to be enjoyed. 

B. Attitudes toward others such as: 

1. Trust in the basic good will of individuals— prior to the exami- 
nation of motives. 

2. Reacting to others in terms of understanding rather than stereo- 
types. 

3. Willingness to accept personal responsibility for making sodal 
groups work. 

4. Identification of individual or family with the community. 

5. Desire to establish empathy with individuals who hold oppos- 
ing points of view. 

6. Desire to find agreements by compromise on nonessentials. 

7. Willingness to let others take precedence at times for the good 
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oE the group, to allow them to play roles for which they are 

, SaJcriortht other person’s point of view (esthetic, poiit- 
ical, moral, etc.). 


a Attitudes toward society such as; 

1. Recognition that social change is inevitahle but not necessari y 

2. BelieUn the right of ail men to life. liherty, and the pursuit of 
happiness. 

3. Tolerance for ways of life other than our own. 

4. Respect for the dignity and worth of the individual. 

5. Constructive discontent with social ills. ^ 

6. Pride in achievements of the United States along with recogni- 
tion of its weaknesses. 

7. Cooperation with others in group action through democrat 

processes. u j 

8. Balanced attitude toward values of the new, the old, ana me 

status quo. . i 

9. Reliance upon persuasion rather than force to achieve social 
ends. 


D. Attitudes toward intellectual, rational, and scientific thought proc- 
esses such as: 

1. Systematic reflection accepted as preferable to snap judgment. 

2. Disposition to regard most problems as problems that can be 

solved, but that some are insoluble. ^ . 

3. Recognition of the limitations of knowledge and appreciation 
of the functions of faith. 

4. Recognition that learning is a continuous, lifelong process. 

5. Respect for expert judgments combined with the desire to 
\'alidate these judgments against experience. 

6. Willingness to reserve judgment but also to act when action is 
required. 

7. Responsibility of the student to take initiative in the learning 
process. 

8. Making practical applications of what is learned. 

9. Intellectual curiosity and honesty. 

10. Faith in tlie possibility of communication with others and a 
recognition of the limitations of language. 

11. Desire to relate all of knowledge into a systematic and coherent 
pattern as over against compartmentalizing knowledge in dis- 
aete areas. 

12. Disposition to regard the solution of problems as relative to the 
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cultures and epochs. 

E. Attitudes toward ethical ^“"dards and valu« of our culture. 

(No topics were listed under this heading.) 

During its first long “feting in f '-bntties^for 
tudes Committee was faced wi P attitudes 

activity. Attempting a study with resp 

.rts .111, -““S, t -s'” 

ever, consideration of any g 

skew the work of *e committee in such^ 

relationship it possible to evolve a study 

lege committees might be lost. ^ F ^Moirintr instruments, 

which, although demanding attitudes°of interest to 

would clearly impinge on a maj y instruments dealing 

the committee? Was it possible p P intercollege 

with traits so broad as to be of concern to the other m 

committees? . ffi,.,nativelv the committee pur- 

In answering these Resulted in two dis- 

sued two parallel lines of ^ instruments were used 

Crete, but interacting, instrumen^^ although 

in two independent clusters of .„a;(.i.ntly unique as to war- 
serving to illuminate each other, are sufficient y q 

rant separate treatment. 

«ir THF Inventory of Beliefs 
The Development of the invilin 

1 I, Id mav be regarded as reflections 

The opinions which people ho J 

of their attitudes, values, and a justm a^ong which 

they are responses and feelings, jp of reaction to self, to 

helps to define *1 “m another point of view, 

others, to institutions, and to lU . as generalizations 

many statements of opinion may e B widely shared by 

base^d on facts, principles, or “"“f "taW^or edVcation) but 
educators (and may be regarded 1 ^ matters of belief 

which at the same time are stil 8 person who pos- 

and individual interpretation by the layman or pers 
sesses less breadth of understanding an i 
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The studies of many psychologists over the past ten years have 
provided an encouraging demonstration of how opinions can be 
used as indicators of basic personality structure Many useful rela 
tionships and concepts have been put to empirical test The volume 
on The Authoritarian Personality^ is especially fruitful 

It appeared promising to attempt some kind of merger of the 
insights of ps)chologists regarding personality patterns and the in 
sights of educators regarding the behavior (broadly viewed) which 
should characterize the product of general education Such a 
merger would attempt to express the relationship between these 
psychological and educational interests Instruments resulting 
would be concerned with the differentiation between the mature, 
independent, reality minded, flexible, adaptive, secure and com 
fortable individual who is seen as the potential base and antici 
pated outcome of a program of general education in a free society, 
and the childish, self-centered, threatened, aggressive, rigid, com 
pulsive, insecure, and uncomfortable individual whose concomi 
tant attitudes and values are seen as essentially antidemocratic and 
in opposition to the objectives of general education 
The fundamental assumption underlying such a scale is that the 
objectives of general education can serve as a base from which may 
be inferred the model organization characterizing the personalities 
of those most adaptable to the purposes of general education An 
attempt should be made to explore the manifestations of the anti 
democratic-democratic continuum as relevant to the problems of 
general education in terms of the individual s relations to (1) ideas 
and intellectual abstractions, (2) social groups and identifications, 
(3) interpersonal relations, and (4) the self 
These four levels of personal involvements are seen as indexes of 
t e ind of psychological maturity which are significant for general 
education By investigating these four levels m the light of under 
png psychological variables or dimensions, information may be 
obtained concerning the basic personality structure of the indi 
vidual Re-examining these four levels m terms of the expressed 
contents of each provides information on the altitudinal onenta 
tion and leiel of attainment specific to each content area In addi 

195m 'tuthontanan Penonahty (New York Harper & Bros, 
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tion, there might also be the possibility of collecting data relevant 
to such matters as logical thinking, consistency, evasiveness, and 
possibly others. 

The formal structure of such a scale was expressed tentatively 
as follows: 


Level 

I. Ideas and institutions 
(ideo centrism) 


2. Social groups (ethno- 
centrism) 


3. Individuals, interper* 
sonal relations (socio- 
centrism) 


Content 
1.1 Philosophy 


1J2 Religion 
1.3 Arts 


1.4 Sciences 
13 Politics, economics 


2.1 Out-groups (Negroes, 
Jews, other ninorJlies, 
foreigners) 


2.2 In-groups (American- 
ism) 


5.1 Family (parents) 

3.2 School (teachers) 

3.3 Church (ministers) 

3.4 State O’ublic ofRdals) 
33 Business and consumer 

relations (tradesmen) 
3.6 Friends, peers, siblings 


Variables or Dimensions 

1.11 Materialistic, manip- 
ulative, power, cyni- 
cal 

1.21 Mystical, ritualistic, 
nonpersonal 

1.31 Romantic, antisen- 
sual, anti-intellectual, 
anticultural 

1.41 Application, limita- 
tion, antirational 

131 Dependence, adher- 
ence to ovtwoded 
ideas, distrust, denial 
of conflicts 

2.11 Personal cbaracteris-. 
tics (oifensive, imma- 
ture, threatening, in- 
trusive, sedusive) 

2.12 Solutions (pseudo- 

democratic: segrega- 
tion, limited partid* 
pation; antidemo- 
cratic: elimination, 

exclusion; insoluble: 
fatalism, despair, cyn- 
icism. 

221 Uncritical acceptance 
of values, exclusive 
pride in member- 
ships, blindness to or 
dismissal of short- 
comings 

3.11 to S.I6 Irrational ac- 
ceptance of external 
authority, unwiliing- 
ncss to assume per- 
sonal responsibility, 
shift of responsibility 
to others, blaming 
others for failure 
(extra-punitive), re- 
sistance to departure 
from tradition, de- 
personalization of re- 
lationships, sentimen- 
tality (momlsm, etc) 
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Level Content Variables or Dimensions 

4 Self (ejocentrum) 4 1 Self concept self eval 4 11 Perception of extcr 

nal world as threat 
emng as manipulat 
able submission and 
aggression rigidity 
and compulsion su 
peistition and stereo 
type destructiveness 
and cynicism free 
floating anxiety pre 
occupation with 
health and sex over 
spiritualization de 
mat and fear (self 
others) depersonal 
ized sexuality 

To implement this conception in the form of a test a number 
(some three thousand) of cliches pseudo rational statements or 
inappropriate generalizations were collected From these were 
!« comprise a trial test form Each statement 

(nnn'i ? judged to be revelatory of some of the four per 
^ ^^^usions listed above Insofar as possible each statement 
^ capable of being judged as a point of fact if sufficient 
letters available For example the statement Sending 

_p congressmen is a waste of time, is at once an indication 
be uation of authority and a statement which can 

ilarly the by recourse to appropriate evidence Sim 

of htic,« ’ predictions of economists about the future 
expression^ better than guesses is at the same time an 

contrarv V **^ationality or anti mtellectuahsm and a statement 
contrary to known fact 

data 1 9n expert judgment and statistical test performance 

a final fo f J^tements were finally selected for inclusion m 
be presented” ” "''^ntory of Beliefs These statements were to 

(1) S rTneh by a four element hey 

W Strongly agree, (2) Agree (3) Disagree, and (4) Strongly dis 

to be itemrwul'”"'r" u intended by the committee 

was based uoo ' ” should disagree This limitation 

Tre m" liT™ : toTnX“"“'^'="‘'°“ 

to nnd than were agree items Our everyday 



PERVASIVE OBJECTIVES 2: ATTITUDES 219 

speech, newspapers, and light literatnre are replete with gl^ 
generalizations which have just enough truth about them to make 
them appear plausible, yet which would be rejected on close 
analysis unless some inner compulsion required their acceptance. 
Statements such as, "If you want a thing done right, you have to do 
it yourself.” or “Man has an inherent guide to right and wrong- 
his conscience.” are part of our culture. It is much more difficul 
to find short statements, capable of eliciting the same order of 
response which are not either banal or hopelessly complicated. 

Second, the fundamental purposes of the Inventory of Beliefs 
could best be served by posing items with which mature indi- 
viduals should disagree. Particularly as research was earned on 
with trial forms of the inventory, it became apparent that the in- 
ventory had its greatest utility in isolating a homogeneous group 
of individuals representing the antithesis of the mature, adaptive 
individual desired as a product of general education. It seemed 
that the identiheation could best be obtained by presenting only 
those statements to which but a limited number of persons would 
agree. Items which could be accepted by a number of less rigid 
individuals were therefore not employed. 

Still a further reason for including only disagree items involves 
the way in which individuals perceive a given cluster of objects. 
If individuals, to use an analogy, are given certain objects of black 
and of white color, they have little d.fliculy m disanminating 
between the two, even though the black and the whites may re- 
spcctively range in tone. The black-white dichotomy, however, 
sLes to preclude discrimination of tone within e ther of the two 
principal colors. However, students presented only the black ob- 
jeets without being given the white ones f- ^nce are more 
easily led to discriminate among the various tones represented. 
This phenomenon appears to be as the 

Inventory of Beliefs. Students presented with only disagree items 
have no "white" reference ‘’"'^""■nate be- 
tween die various disagree items and those 

which they might reject. If some agree items are intermixed with 
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plified by the disagree items, it seemed best not to give students 
the reference point of even a few agree ones 
Another testing technique involved the four element key rather 
than an odd numbered one In the final analysis, the committee 
was interested only in whether or not individuals accepted or re 
jected Items However, some examinees find themselves frustrated 


by simple dichotomized response keys of agree disagree To ac 
commodate their feelings and in view of the fact that no ambiva 
lent response was provided, gradations of agree and of disagree 
were employed The use of such a key has considerable support 
from experiences of researchers using forced choice techniques It 
further has the support of Sanford and his associates,* who used a 
SIX element key, still avoiding the neutral position on their F scale 
which is in a sense a prototype for the present instrument 
The final form of the Inventory of Beliefs consisted of 120 state 
ments distributed among the four dimensions of ideas and institu 


tions, social groups individuals, and interpersonal relations, and 
se These four qualities were represented by items of concern to 
the content of the several general education areas Those items 
ea mg with ideas and institutions would have relevance for 
genera e ucation courses in the humanities, sciences, and some 
social sciences The social groups items and the individuals and 
mierpenonal relations items would have primary relevance for the 
la sciences Items dealing with the self would have particular 
^spects of the usual social science offering and a 
° evance for courses such as Effective Living or Hu 
tinn^ ^development Each of the subject matter onenta 

\arnhl ^ counterpart in some psychological 

dimension of personality Those items of the ideas and 
m->» category which dealt with philosophy would represent 
"^^^^ipulative, power-centered, or cynical onenta 


mp-ininrr ^ ^^^^pouscs to all of the Items were conceived to have 
inf*^ Personality structure The individual scor 

wis ronrp A ** accepting a large number of statements, 

itanan ^ being immature, rigid m Ins outlook, author 

itarian m his relationships svith others, and compulsive in many 

* In Adorno eta! ibid.p 59 
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of his actions. The person scoring high on the test, that is, rejecting 
a goodly number of statements, tras conceived of as being adaptive, 
flexible, mature, and democratic in his relations with other people. 

In addition to the total score, individual items were considered 
to have significance as discrete expressions of opinions and to re- 
flect level of achievement in specific general education courses. 
Teachers of humanities, for example, might be interested in know- 
ing the proportion of a student body who believe that "Reviewers 
and critics of art. music, and literature decide what they like and 
then force their tastes on the public.” Science teachers might e 
interested in the opinions of students on the statement, ‘ Science 
is infringing upon religion when it attempts to delve into t le 
origin of life itself." All teachers might find it not only interesting 
but profitable to know how students feel about the statement, 
"A lot of teachers these days have radical ideas which need to e 
carefully watched." On the subject-matter side, one 
social scientists probably emphasize to the eclipse o a o i 
is the interdependence of individuals and society. Students re- 
sponses to the statement. "Each man is on his oim in 

determine his own destiny,” could reveal how sureess u teac e 
have been in getting their point of view across. e uman 
justment teachers who constantly emphasize the practical and ther- 
apcutic values of conversation might like to noiv i stu en 
believe, "IVe would be better off if people would talk less and 

would work more." r 

While this particular form of the Inventory of e le tepr 
the final development of the test by the Attitu cs 
many of the insights concerning it and leads to urt icr 
with it have stemmed from modifications of t e asic m\ 
by George G. Stem and his associates at tlie University of 
Stem has constructed two other forms of the in\cmor>, ^ 
and Form T, each of which used a large nucleus o items r 
original inventory plus other items cither to e “^tee ' 

items involving another dimension to be used in exp . 2 

of the responses obtained to the items used from t le . . 

The !20 item form of the test has the disads-antage ^ ^ 

betsveen SO and 15 minutes of chass time 

apparent tint all of the 120 items were lughly interrela.ed, this 
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interrelationship suggested that the test might be divided in half 
by some rational means so that two comparable 60-item inven- 
tories would result without appreciably lessening the reliability 
of the technique. Such a split was made on the basis of item analy- 
sis data and original item affiliation to the various dimensions. 
Sufficient experimental work has been completed to warrant the 
conclusion that individuals wanting a shorter form of the test 
could use either group of 60 items with impunity. The precise 
assignment of items to these two forms is indicated in an instruc- 
tor s manual for the Inventory of Beliefs.* 


Inventory of Beuefs Research Studies 
The Inventory of Beliefs has been subjected to considerable 
research as to its technical effectiveness. Form I of the inventory 
sufficiently reliable for purposes of group or 
^measurement. Some thirty-odd reliability studies have 
n made resulting in coefficients ranging from .68 to .95 with a 
lan coefficient of .86. Even the two shorter forms of the test 
nor reliable as the original, although they should 

not be used for individual diagnosis or prediction. 

severd Inventory of Beliefs has been studied in 

to vf'TiPr\ Its validity as a list of statements of concern 

oDinion nF presumed on the basis of the expert 

which thf 't ^ 'vho prepared it and of the sources from 

submitn'n ^ niaterial was derived. This was corroborated by 

one frrnitn t ^ of faculty members, asking 

"ideTr’n^A° terms of how it would want 

wasa^ltM*! S^^^ral education to respond. The other group 

field woiilH°# whether or not expert opinion in the relevant 
from thecp to support or to deny the statement. The results 
judves coulJ*^ revealed that a large majority of the faculty 

obkctive^of.^'’; ‘he 120 items as relevant to the 

list comisteH^ ^ and that three-fourths of the entire 

esmbtished h, fa^^e^ 

y of the test as providing an index of personality 
•Princeton. N.J.: Eduation,! Te.Ung ScrWc*. 
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structure was assessed in two principal ways studying various in 
tertest correlations and studying characteristics of persons whose 
scores fell at the two extremities of the range Table 52 presents 
some intertest correlations with other measures used in the Co 
operative Study 

TABLE 64 

CoEmciENTS OP ConnELATiON OF TIIE Inveotobt op Beliefs with Other Tests 



CocmCIEHT or ConREUATIOH 1 


No OT 

Tmt 

Lowest 
t in Any 
College 

fi ghest 
t m Any 
College 

Arcrnge 

t* 


STOTurra 


Pte Test 


ACE Psychological Examination 
Cntical T hinkin g Form A 
Critical Thmkmg m Social Science 
Problems m Human Relations 
Critical Thinking Form B 
Cntical Analysis m Readmg and 
Writing 

Science Reasoning and Under 
standmg Form A 
Science Reasoning and Under* 
standmg Form B 
Humamties Participation Inven* 
tory 


ACE Psychological Examination 
Cntical Thmkmg Form A 
Cntical Thmkmg m Social Science 
Problems m Human Relations 
Cntical Thmkmg Form B 
Humanities Participation Inven 
tory 

Critical Analysis m Readmg and 
Wntmg 

Science Reasonmg and Under- 
standmg Form A 
Science ^asonmg and Under 
standmg Form B 




26 

12 

2,171 


43 

25 

12 

2,171 


46 

SO 

12 


-- 08 

14 

49 

37 

38 

29 

11 

5 

1,853 

641 

07 

87 

24 

8 

1,897 

16 

46 

31 

8 

1,010 

07 

38 

21 

6 

639 

- 06 

16 

10 

3 
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Post Test 

12 

05 

24 

36 

12 

43 

39 

63 

47 

28 

21 

22 

35 

45 

19 

5 

5 

5 

4 

4 

743 

743 

743 

505 

462 

17 

27 

24 

2 

340 

04 

31 

21 

6 

994 

25 

61 

32 

5 

864 

24 

40 

35 

2 

166 


•QuumMcNenur PjycioIofft^oJ Staiwiw* (N®* V«k Jobn Wiley & SonJ Inc„ 1919) pp UM 

These data emphasize the aftecuve quality of the 
The correlation coefficients between the inventory an e 
requiring critical thinking are generally quite low as 
with similar statistics showing the relationship etween 
critical thinUng ability' The highest coefficients obtained 

*Sec chap 7 
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between the Inventory of Beliefs and the Problems in Human 
Relations Test— both products of the Attitudes Committee and 
both possessing considerable overlap in respect to the traits in 
volved The coefficients between the Inventory of Beliefs and the 
American Council on Education Psychological Examination were 
generally quite low 

Low scores on the Inventory of Beliefs might be presumed to 
represent an atypical segment of a college population This pre 
sumption led to inquiry with respect to possible differences be 
tween very low and very high scorers As a result of such inquiry 
a considerable body of data has been accumulated which either 
irectly supports or does not contradict, the conclusion that the 
mventopr does measure certain types of personality structure Be 
cause 0 limitations in space these data cannot be presented m 
demd. but they have been presented elsewhere « 

generalizations which follow have been sub 
u ^ J roughly parallel studies in two institutions, as well 
arp j replication undertaken by one of them They 

dents \ three independent samplings of stu 

pgn 1 ations of 200 to 500 The samples upon which the 

who based represent students from these populations 

also eauntf’^ / extremes of the inventory distributions and were 
from Z9 to 61 fe ea'ch 

latl'm Chara«e‘’nzed'’by7h''‘^ ^'7' 

comnarisfins ^ score range on the imentory, 

differences Se widely separated do not reveal sudi 

Tourh^L have been obta.ned 

,, xpenmental forms of the inventory (Forms 

l George Stern et al a* ^ ' 

III Univenity of Chieagi II [etc] (Dittoed Chicago 

2 S Goldberg and C Stem '^'Th- a ». 

lion " paper presented at GOih a Penonality and General Educa 

Assoaation September 19o2 Wati, Comention of American Psychological 

5 C St.m "NewT,chi;,JI fa7'“" 

C/iiciigo/ January I 953 ^ testing Coffeg^ Faculty Bulletw, t/niuersity of 

< G Stem “Studies in Pcrso t 

paper presented to the Midwest™ P The N R and S Syndromes" 

5 C Stem “Personality Centered 2’^°*°S>cal AssoaaUon May I9j3 Chicago HI 
presented at 61st Annual ConventTn^ Psychological Unification paper 

her 1953, Clescland Ohio, abstracted. Psychological Association, Septem 
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S and T) These forms provide indexes which oSer greater ^lagnos 

tic precision insofar as personality "VTfLm of th 

othenvise do not appear to differ from the f ™ 

inventory All three forms contain 60 common items, and 
developed by the addition of 40 new items to make a total of 100 
m each test These new items provide a basis for ^ ^ J 

(both agree and disagree responses) in contrast to the umdirec 
tional scoring of the parent form (all disagree 
abilities of all three forms are uniformly high, as previously noted 

"“anTdiSerences (level generally 001 or les^^are found 

mg the high scorers with more high scorers enter 

inanities, social sciences and English, rices however, neither 

these fields than do low scorers In the „h,ch they 

group IS distinguishable from the general populations from wm y 

were drawn 

Achievement data avprflMs 

Test performance -Mthotsgh differences in 

are negligible, significant difterenres j,£ the^soaal sciences 

found in comprehensive examinations in tn 
and humanities 

Withdrawals -A significantly largCT ^ 

to have withdrawn from school by the en o y 

Interests and attitudes . , flipir 

High scorers were found significantly *'■6'’” io,„er for 

scores for artistic, musical, and social scienc ’ S 
computational and clerical f^,"es were fLnd The high 

Significant differences m vocational exoressive, or ab 

scorers prefer occupations ‘"™'”"|,*"‘"^rtheoreticS physics The 
stract activity, such as ' ete or status oriented voca 

low scorers prefer more impersonal, co 

tions such as engineering medicine, found, the low 

Significant differences in religious a orthodox or fundamen 

scorers being more likely to be associated with orthodox 
talist groups than are the high scorers ,r,^, rates Inch scorers 

Analyses of data from an ucuviues mi ento^ indicates 

seem to prefer activities which reflect crests, and esthetic ex 

bailor, abstract and analytical their preferences reflecting 

penences Low scorers reject such activiti , P 
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onentation toward the achievement of finanaal status, security com 
puisne orderliness and submissive or dependent behavior 

Consistent with these findings are the results from several other 
studies At one institution teachers and students were solicited for 
Ktimates of students possessing the greatest degree of leadeiship 
These leaders made significantly higher Inventory of Beliefs 
scores than the student population at large In another study the 
popu ations from a large Northern and a large Southern university 
were compared The students from the Northern university 
achieved better on the Inventory of Beliefs than did their Southern 
counterparts However, when Southern students of high aca 
emic aptuude were compared with similarly qualified Northern 
ents t e reverse obtained Southern students of this category 
ppeared less authoritarian than did Northern students Perhaps 
emtrf P<>“‘bility that intelligent South 

of tho environment overcompensate for those aspects 

■ty grou^ clutiate which are restrictive for some minor 

‘he Inventory of Beliefs yields a 
to other fi r "'h“^h shows critical relationship 

proved Lt " «Per.mental work with the instrument 

entire ranvc *'"ee these studies were based on the 

analogy 35^51^113^1^ he conceived of by 

selected m-mm a '’anous medical tests which identify only a 
dt The chi xLy 

of the ponulatinn 1. between the small percentage 

majority who are pathology and the large 

ferentiates between inventory dif 

scorers are the individuals°lr”'"^ 

common complex of traits " P''“‘^'’‘ ® somewhat homogeneous 

Pre “’’d Pest Test, 

The intentory of Delict. 1, 1, 

and 1 post lest basis to student! !!!.'" on a pretest 

gams on the test made over ^ "“"’hor of colleges The 

53 for each of these institutions recorded in Table 
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The two schools whose students made the greatest gains (1 and 
2) are schools whose student bodies are highly selected and whose 
programs of general education are carefully integrated. Classes are 
conducted with considerable emphasis placed on small discussion 
sections in which the teacher is considered as only a moderator. 
Whether such factors are operative rve cannot tell. They do, how- 
ever, take on added significance in this regard when the gains of 
other schools are examined. Students from School 3, for example, 
have approximately the same pre-test score as School 2, yet there 


TABLE S3 

Meah Gainb of Stodents on Inventort of Beliefs Post-Test, 
CLAsaiFTEO AccoRDma to Pbe-Test ^andino 



is a marked difference in gains. School 3 is a private institution 
whose students may or may not take the general education courses 
which are organized into fairly large lecture sections with little 
attention paid to articulation of the course content or conduct 
with the rest of the campus life. 

Schools 5 and 7 demonstrate other possibilities. Both are state- 
supported, land-grant colleges appealing to approximately the 
same kind of student. School 5 has had a limited program of 
general education, whereas School 7 has had a program well de- 
veloped and required of all students in the institution, regardless 
of their eventual field of specialization. 

Students at several institutions were tested with the inventory 
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TABLE 54 


Mean Gains of Sttoents Tested bt the Iwv’evtort op Beliefs 
Three Tbies over a Two Year Teriod 


TiSTINO DiTB 

1 CoLuas A 

I College B 

N j 

Mean 

SD 

N ; 

Mean 

SD 

Fall 1951 1 

Spring 1952 

Sprmg 1953 


56 53 

61 65 

62 11 


24 

24 

24 

63 08 , 
70 25 

72 S3 1 

11 29 

14 41 

11 88 


three different times-at the beginning and end of the freshman 
year and at the end of the sophomore year. These data, illustrated 
in a e 4, reveal the pattern found elsewhere of a markedly 
lower gam for the second year. 


TABLE 55 


lOX 


pT»r Tpo-n n IN STUDENTS BELIEFS 01 

Test and Post-Test of Inventory op Beliefs 


obs.S-.i,o7“ ‘l-l » »«l dependant apoa “ 

Umt dealing with mychdoot, 

puSshed"ev2ely should be 

»» ‘'“h 'Wd'ea „ pabl.o 

3 ^ people ^ without hurting other 

dtS?; " ™ <”™ m Wa aad data™™ hu owa 

6 humanities 

29 p literature decide 

29 Boola and movies should Public 

uplifting themes mstead*t?i1.“^'"®^** enterUinmg or 
moral, or tragic ones ° “ “ Present unpleasant, im- 

“to practical use ” “ ®°« who turns theories 

83 Pabl7Sb'''4'fe -aAea^.^ d.agaaa.bla 

web of influence wh^cb caught m a 

■ t-n tenos to corrupt them 


% Dit* % Di*- 
ameins sgteeio; 
FieTest PoatTeit 


49 


18 
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The influence of maturity has been suggested by a number of 
comparisons between matched groups of fteshmen seniors 
and between varying age groups of a freshman 
general, all such studies show that older students rnake significantly 
higher scores than do younger ones. 

of such magnitude as to discredit the fact of differenti g y 

institutions indicated above. , 

It may be of some interest to note the nature of the ^ 

Inventory of Beliefs and how students shift over a P‘=™d o' “me 
with respect to some of them. Table 55 presents 
from the inventory together with a percentage o 
one large population who rejected the item in the al 19 1 and 
the percentage of the same group rejecting the same the following 

spring. 

The Development of the Problems in Human 
Relations Test 

The Inventory of Beliefs was designed primarily “ 
thing of underlying personality structure. is pe , j 
tore it was suspected, would manifest itself m “ 
erent ways, a major one of which would ® ^ dienity of 

individuals demonstrated respect for the wort 
human beings. However, so fundamental to a 
concern or regard for the worth of others that c^mm. e be 
lieved a separate evaluation project was warranted to assess 

presence or absence in students. . j , nmiect 

Consistent with the general mtionale of a«budes, this project 

for the worth and dignity of others appeared to invol 
more specific attitudes, which might be classi e 

1. Those directed toward persons ^ 

a) Toward the views and vocations o , others ^involving 

b) Toward special privilege for oneself and others (i 

c,S“hre^qi'afltf of various racial and socioeconomic 

d) & the rise of minority groups (including die concept of 
protection for their rights) 

2. Those directed toward processes 

a) Toward the concept of cooperauon 
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TABLE 64 


Mean Gains of Stotents Tested bt the Ln’Evronr of Beliefs 
iiiREE Times over a Two Year Period 


Tisirao Dat* 

1 COLLEOK A 

1 College B 

N 

Mean 

SD 1 

N 

Mean 

1 SD 

Fall 1951 

Spring 1952 

Spring 1953 


66 53 

61 65 

62 H 


24 

24 

24 

63 08 

70 25 

72 S3 

11 29 

14 41 

11 83 


three different times-at the beginning and end of the freshman 
year and at the end of the sophomore year These data, illustrated 
n a e , reveal the pattern found elsewhere of a markedly 
lower gam for the second year 


TABLE 65 

trai AUD Post Test oe Inveetoet oe Beeiees 


% Dll % D» 
tgttt ug tgrtt og 
FoftTeit 


.b'.S.aC"“ dependsEl „p„„ 

jlTJtf'"’ P’ychohis 

pShcS«v"J.l“ “ *« .»d shoPld b, 

“Xol‘SX,“"‘* *» t'"* dvldrcn as public 

people mmion dollars without hurting other 

dttmT “ " “-t determuie his own 

what they like and then literature decide 

Boola and movies should start on the public 

uplifting themes instead^tf entertaining or 

moral or tragic ones present unpleasant, im 

//ms dealing vnth natural snenee 

which many peopKlSdf to describe things 

The scientist who really cotf>a through common sense 
mto practical use “ “ ®°« who turns theories 

i'S ""“1 •cine 

6l03t Negroes would faponr^^ . 
d not kept m their place ®'^*^**eanng and disagreeable 

Public officials may irv to u 

web of mfluence which caught m a 

^«s to corrupt them 


68 

84 


64 


73 
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b) Toward acceptance or rejection of evidence 

c) Toward the values of free discussion 

The committee believed that student feeling for or against 
these objects could be assessed by means of situational type objec 
tive test questions and that student responses to such questions 
would provide an index of respect or disrespect toward human per 
sonahty possessed by individual students The situations included 
a variety of conflicts which arise in everyday living, resolvable m 
any of several different ways The committee hypothesized that 
t e way students met such conflicts would be based upon their 
atmude toward the parties included in the controversy 

o ensure that situations appeared plausible and natural, stu 
ents in c asses of the participating colleges were asked to describe 
which they had witnessed within the past week exem 
dpni' respect or lack of respect for human beings Stu 

morps varied from reflecting only a superficial concern for 

issues insight into the kinds of 

resDonsM w ^“^ntally divide people The more promising 
muted to nth‘ P'^°'^'^ms which were then sub 

individuals of students who were asked how they, as 

individuals would resolve each one For example 

the scnior*wuh?he°hiehe “'Punment to grant an award to 
to wm this award grades Because of her intense desire 

be done about Bettyf'"'" ^ m examinations What should 

which*' toils* tor 'object''""*'’"* material out of 

These responses apS Jr 

points of view from , l, n general five fairly definite 

to interpersonal situatmm Tr*'‘r'* 

titles and described as tollowT" **'’" assigned descriptive 

I Hard boiled Autocrat -Th, u .. 
must constantly check up boiled autocrat believes that he 

the orders and others cam H^^P '*’‘"5" going He gives 

to get conscientious perform ™ believes that the only way 

immediate acceptance of all^”^ ^ to expect and secure discipline and 
ployees, his wife, his children^ “ careful not to spoil his em 

praise He pays his employees and ,h“, 

r 7 and that is all they need He supports 
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his wile and children. It is their place to carry out 

ouestion or always understand them. He is very conscious o£ his posi 

don and authority and believes that others cannot be trusted very long 

on their initiative. , , 

2. Benevolent Autocrat.-Tht benevolent autocrat would bearded 
to realize that his techniques are autocratic. In contrast *e ha 
boiled autocrat, he is interested in his employees 
He wants to see them happy, praises them as f f ^ 
is seldom harsh or severe, and likes to think that he is P S 

happy family group. He urges them to bring their P^^ems to him and 
is imerested in all of the details of their work. AcWa^ he Vade 
benevolence for loyalty. The crux of his uutoCTag lies in *e techn^ 
by which he secures dependence upon himself. He says, P 

the back, "Thafs the way I like it. ... I am glad 7°“ 
way. . . . Thafs the way I want it done." or That ‘ the 

you to do it.” In this way he dominates them y nia ® ^ 

Lurce of all standards. Any failure to live up to 
receives with hurt surprise and intense anger as inHphted to him 
him. He is also very protecting and makes them further 

“'ruLez FtuVe.-The leisse.laire personality 
no confidence in his ability to handle • -ir paper 

play the role of leader. As a supervisor, he may 
work and stay away from those he is supervising. . ^ j, • 

one who helices that to be a "good fellow” h 

able to destroy the cordiality of any mterpersona r leaves too 

unable to be aggressive in any interpersona re a i ■ .yhich they 
much responsibility with others: sets no clear goa s o arrive 

may work; is incapable of making decisions or helping the gr p 
at decisions; and tends to let things drift. endeavors 

4. Demorraa-c.-The democratic personality ," 3 " f^^^ily, 

whenever possible to share with his group (stu en P assi^ment, 
committee, etc.) the decision-making about work p . t^e 

and scheduling. Where a decision must be raa e ^ y • tp 

group to understand clearly the basis of as 

develop as much participation, opinion-givmg, work on 

possibfe, and a feeling of responsibility for success^ of *e woj^^^^_ 

the part of everyone. He is concerned that ea praises and 

stands his role and has opportunities for success 1 * j, Results and 

criticisms are always delivered objectively me 

never personally in terms of what he may or pro- 

courages ivorth-while suggestions and the deve op 

5. Resort to '‘Experts” or Other might be ex- 

responses classified in this category are the ki 
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pected of an authoritarian personality, some are what might be 
expected of the laissez faire personality, others are what might be ex 
pected of the democratic personality who has respect for the knowledge, 
opinions and skills of the expert It might also be expected that the 
o\erdependcnt and immature individual will choose a large number of 
responses in this category Experience may eliminate this as a category 
or further refine it 


Sixty situations were selected and prepared in the form of five 
c Dice multiple response questions, with each response being 
esigned to parallel some one of the five personality types Thus, 
persons judging each of the sixty test items would classify them 
se ves as either hard boiled autocrats benevolent autocrats latssez 
aire personalities democratic individuals, or one making frequent 
resort to expert authority From these sixty items administered to 
trge groups of students, the committee selected thirty Items for 
c usion in t e final form of the Problems in Human Relations 
tho fi, contained responses representative of several of 

^\’li However, the category m which the committee 

cations democratic one The other classifi 

tions anH ^ ^ rational foils for the objective test situa 

not subsmh for explaining why individuals did 

cerned hut r^i7 i the trait with which it was con 

point of Mew iiasappropntm ""'•democratic 

couniLcrm ob^tlc^mTh” "" 

To write alternativeii clearly 
but not lutomaticaHv r, 1 

was a most dilficult tisfi Th ’T “ socially approved 

sponscs was so difficult f""' '*>at disguise of the crucial re- 
tlnn It might otherwise hi P™’”‘’'>' '""de the test less effective 
m this rc<xtrd may he '' "P*"' difficulty of the committee 

It prepared the Invento°"'Tn”* relative ease with which 

cause of it In 7is same ^ effective, probably, be- 

quently appeared in their 

highly unsophisticated This a™ “ 

tncnL further served to weaken the instru 
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Illustrative of the items included in the final form of the test 
are the following: 

1. Tom and Bob who know each other only slightly were double- 
dating two girls who were roommates. A sudden storm made it 
impossible to go to the beach as planned. Tom suggested going 
to a movie. After making the suggestion, he realized Bob was 
without funds. 

As Tom, what would you do? 

1. Pay for the party. 

2. Lend Bob money. 

3. Leave it up to the girls. 

4. Get Bob to suggest something. 

5. Apologize to Bob for making the suggestion. 

2. Your social organization has pledged a student who is not liked 
by some of the members. One of your friends threatens to eaye 
the social organization if this person is initiated. at wou 
you do? 


1. Talk to your friend. 

2. Do not initiate the prospective member. 

3. Get more members to support the prospective 


member. 


4. Vote on the prospective member. 

5. Postpone the vote until the matter works itself out. 

While most of the situations deal explicitly with conflicts with 
which college students might be familiar, an attempt was ma e to 
involve various kinds of minority groups toward which un 
cratic attitudes might be manifest. There arc items dea mg wit 
Negroes, very young children, criminals, Jews and Chinese, run 
ards, neurotics, and the insane. These are the persons against 
whom prejudices are frequently directed and are the in ivi ua s 
most resented by the extra-punitive authoritarian persons o con 
cem both in the Problems in Human Relations Test an m 


Inventory of Beliefs. . ^ 

The scoring key for the Problems in Human Relations ^ es 

prepared a priori on the basis of the face validity of t e siti 
and the alternative responses. The reliability of the ro ems 
Human Relations Test using such a key proved to be ina cq 
Other data, however, accumulated during the ear y p lascs o 
search activity, suggested that the test had possibilities for ic p 
poses for which it was intended. With this in mind, the comm 
searched for another method of keying the instrument wliicn migi 
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result in greater reliability Since the best judgment of the commit 
tee members had been expended in preparing the original key, 
empirical means appeared to be the only solution 

Rejecting for a time the entire original key, the committee re- 
vieived each item of the test and selected only those situations 
which possessed one very clear cut response indicative of the demo- 
cratic category These few responses were combined into a key and 
papers scored The upper and lower 27 percent of the papers based 
on this short key were selected and item analyses made of every 
single Item in the test Those items showing high coefficients of dis- 
crimination were selected for inclusion in another key The reli 
3 I ity of the test was then computed on the basis of this second key 
and was found to be slightly increased The process was repeated 
and new coefficients of discrimination computed for every item 
^in t e Items possessing the highest correlation coefficients were 
inclusion m still a third key, and again reliability co- 
fc'ealing still more increase This process 
conditions became apparent (I) a drop m 
in a i * ^ change m complexion of the items included 

geneity^ o°^ogeneuy, as judged by inspection, to hetero- 

mnn analysis of the democratic category, similar 

dimensions of the test These 
ticallv ^ benevolent autocrat category was statis 

found^:IthTeU7m "" 

analjses of the^nonH authonty category These 

clcarlv emnTiaf aspects were done, it should be 

of subscorc^s, bm with” intention of developing a senes 
ot the m,jor or democraucTale'’'"^'’"' 

ttiriom Undl M subjected to 

■ng .mtroment, and has ahf h effectiveness as a measur 

tlie chances whirh courses to determine their status and 

.nditjel^hlthaS^ ^ 

teas slightly lower tha J m n' T'^ 

G1 to W Hr"el- uie re. ?r' ”"^"3 from 

KuderRicliardson approach of r 

PP oach, of the final, statistically developed 
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key was .82, which suggested the feasibility of using the instrument 
for purposes of group measurement. ^ 

Item analyses of the test indicated that the coefficients of dis- 
crimination were generally quite high, much higher, in fact, th^ 
one normally would expect. Part of the magnitude of these coeffi- 
cients is explainable by the fact that the test was a relatively short 
one, giving each item greater weight in determining the final 
score. The difficulty of the items was also satisfactory, but exhi i- 
ted, however, a slight tendency for the items to be a little too 
easy. This evidence underscores the point previously ma e, t at 
the correct or democratic responses were, for the most part, air y 


obvious. 

The validity of the instrument was studied from several ditteren 
points of view. Face validity, of course, is the basic consi eratio 
In addition, various groups of students possessing relevant trai 
as judged by other means were compared with respect to scores on 
the Problems in Human Relations Test. In one stu y stu en 
were judged by teachers as to their possession or lack ° 
of respect for the dignity and worth of human persona ity. p 
sumption of validity was supported by the generally ig er sco 
on the Problems in Human Relations Test made ■* ^ 

to possess the trait than those made by persons ju ge as ac 'i 
the trait. In another study, student leaders were shown to po 
the trait to a much greater extent than the student popu a | 
large. Still a third study revealed that in general older mdividua 
made higher scores on the test, . . _ 

One major concern of those who built the test ' i^finns 
relationship between scores on the Problems in Human e 
Test and other measures. Tables 56 and 57 

relationships discovered, expressed as correlation coe were 

four groups treated in the tables represent 
each administered one distinct battery of Study tests o 
was the Problems in Human Relations Test. Prnb- 

One could expect to find positive correlations etween 
ferns in Human Relations Test and other lotions 

ever, since tlie responses on the Problems in 
Test were opinions rather than the solutions to in sliould 

ferns, relationships with tests demanding ex- 

be fairly low. The evidence in Tables 56 and 57 verifies tl.ese ex 



TABO: 56 

COEmCIENTS OF CORRELATION BETWEEN PROBLEMS IN HtJUAN RELATIONS PrE-TEST AND CERTAIN OTllSn TeSTS, 1031 
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pectations The generally higher correlations found between the 
Inventory of Beliefs and the Problems in Human Relations Test 
suggest some overlap with respect to the traits being appraised. 
This is, of course, in keeping with the rationale of the two ms ™ 
ments, which conceives of the Problems in Human Relations Test 
as assessing directly one of the basic traits appraised indirectly by 
the Inventory of Beliefs 

Regardless of their institution, female students genera Y 
higher democratic scores than did male students, and male studen s 
made appreciably higher scores in the 'more 
of the test. These differences can probably be accounted fo y 
fact that m our culture, at least at a superficial eve , eni 
generally conceived to be the more considerate, a aptive se 
females would be more readily attracted by the obviously dem^ 
cratic solutions to the problems than would males who, by cu tur. 
definition, are supposed to be more hardboi e ' * 

and egocentric. Whether these differences exist in asic p 

or in the subconscious of individuals needs to e exp 
further research. 

Pre- and Post-Test Results 

Students from a number of institutions were „ 

1951 and again in the spring of 1952. The results of this testing 
are indicated in Table 58, showing how students from ea 

table 58 

Mean Gains of Students ok 

Classified Accobdiko to Pbe-I-est siamdinu 


I IKmAULT 

CoLUa*! txiw Gboup 



1 «7 


1 « ji.w* 
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divided according to their pre test status, changed over the period 
of a year 

From these data certain suggestions emerge The first is a 
replication of the tendency found with all of the Cooperative 
Study tests, that the lower the students pre test score, the greater 
the gain made over the period of a year Again we must underscore 
the point that we have no way of knowing whether this is primarily 
statistical regression or whether it has deeper significance The 
second IS the generally slight gam made by students from each one 
o t e schools represented There are, it is true, some major 
itferences as between the loss of -0 38 for School 6 and the gam 
ot ^ 11 made by students in School 7 However, such gams and 
osses o not fit into any me^jiingful pattern with respect to real 
institutional differences 


Relationship of the Attitudes Committee 
TO Other Intercollece Committees 

nanlv^ Attitudes Committee, while partly successful and 

ZZ oZ ' “ having had considerable 

bee*^ oneLd f Cooperative Study, and avenues have at least 
the SIX Stud “'ure research and educational practice Each of 
the affectivp^ committees came sooner or later to a concern with 
ssted iXdr' They were inter 

thinking and thTothlr furthered or retarded critical 

existence of a «p cognitive traits being investigated The 

energy to the affecr^' “itudes Committee devoting intensive 
teehmqiest ml 'hem to rely on it for 

provided with greatelnllf “'her committees were 

general educatmn The A,.T '"''““Ration of other objectives of 

implied assignment by develoDinl, ”"”"” 

could be considered as relJl!! instruments each ot which 

tive oriented committees Xl. n"" m all ot the five cogni 

Inventory of Beliefs at th. i person identified by the 

of just as much concern for Thl 

ties teacher The trait nf 'eacher as for the humani 

IS an important outcome individual human personality 

come of all courses in general education 
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SlGNinCANCE OF THE ATTITUDES PROJECTS 
If the work of the Attitudes Committee, particularly with re- 
spect to the Inventory of Beliefs, is substantiated by subsequent 
research, a major new conception of education may well have 
evolved. This conception assigns much greater importance to the 
role of personality and to the affective aspects of learning than 
has traditionally been the case. The traditional means of predict- 
ing academic success has been through such things as aptitude tests 
(either psychological or subject) and high school grades or patterns 
of courses taken. While some of these means have been fairly 
satisfactory in indicating which students might expect and whic 
might not expect to profit from college, there has always been a 
large number of students whose subsequent achievement be le 
predictions made about them. Evidence obtained from the Inven 
tory of Beliefs suggests that perhaps identiHcation of students can 
be made on some basis other than intellectual prowess. ^ 
institutions, albeit somewhat atypical ones, the Inventory o ^ ^ 

or an adaptation of it identified approximately 17 percent o t e 
student body who could be expected to have difficulty accommo- 
dating to a general education program. This group of in ivi ua s 
was not an inept one. As a matter of fact, their psychological apti- 
tude test scores were equal to those of the rest of the stu ent ^ o y 
However, with the exception of this common academic aptitu e, 
the group represented a collection of individuals di erent r 
their peers. This group apparently could not profit from 
education courses organized as they were at the three 
in which they had matriculated. Individuals of this 
dropped out of school before the first year was up, or e se e ^ 
frustrating existence trying to reconcile their ° 

college education is all about with the ideas reflecte in 
they attended. . 

The existence of this group of students poses the ^ 

lem for teachers and administrators of what should e 
this fairly large segment of the student population. ^ ptoud 
is to assume that the personality traits characteristic of t gr P 
are so deep-seated and all-pervasive that education 
them appreciably. If this assumption is warranted, co g 
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either prepare specnl curricula applicable to the group, find cur 
ricula such as certain technical ones for which they may be suited, 
or else accept a high mortality rate among these individuals The 
second possible solution is to assume that while the personality 
structure is deeply imbedded in the physical and emotional struc 
ture of the individual, it still may be altered if appropriate tech 
niques are brought to bear While such persons probably react 
negatively to typical classroom situations, there is the possibility 
of reordering some classroom work, emphasizing student counsel 
ing and exercising judgment in providing student living accom 
modations so that these persons may be gradually brought closer 
to the norms for the entire student body Both of these possibilities 
need to be explored for resources of this nation are insufficient to 
permit the luxury of wasting the talents of aiiy able subgroup of 
the student body 

Beliefs has tentatively identified and further 
wor as described one portion of a population for whom certain 
special considerations are suggested It might be possible that there 
are ot er similar elements of the population different in character 
s ICS w 0 might need other kinds of special treatment The re 
arch incident to the development of the Inventory of Beliefs 
ggests approaches which may be appropriate to defining such 


affective domain as relevant to higher 
ethiral major concomitant problems ranging from 

ethical matters to merely items of technique 

not been l^estion which must arise and which has certainly 
m trying whether or not teachers are ethically justified 

affinnative-thaTis*tha™L““"“‘’“ 

they mav comp ns i , to try to affect attitudes- 
in totalitarian of indoctrination found 

which was postulated at thJV“'''’‘ attitudes 

attitude m the final analysis chapter, then an 

Are teachers enufi ^ V w a matter of individual conscience 

Tot 

toward certain pnnctles cor""'. 

teachers who answeTverwi7r 

/ have a difficult time specifying pre 
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cisely at what level they will try positively to affect attitudes and 
at what level they will leave such matters to the individual. 
Teachers denying their right to affect these emotional aspects may 
very well be accused of negativism in not wanting to take a stan . 
and of outright prevarication in denying that they do affect student 

attitudes. , 

If teachers elect to try to affect attitudes overtly, the question as 
to how this may be done arises. Some people contend that atti- 
tudes may be affected best through providing knowledge. They 
would argue that the more we teach students about the wor d in 
which we live, the more favorable will be the attitudes of the e 
students about those aspects of life svhich really count, n o 
nately, research in the affective domain denies t at t is ®PP 
The generally low correlations between attitu es tests 
cognitive ones suggest that attitudes toward an ^ 

thinking about the same matters can develop quite m ep 
of each other. While students gain generally over the P^nod °£ a 
year svith respect to the two attitudes tests this com 
veloped, those gains do not seem to be associated wi g 

in what might appear to be relevant areas of know e ge. 

If attituLs sLm to be unaffected by course 
education, are there ways open to college teac ers sv , 
positive results? In developing democratic sentimen ® ° 
of their students, the implications of practices in t e , jgnt 
professor’s office might do much more toward f 
feeling than any number of lectures about the su j • 

claims that democracy is cultivated on college campus ■ 

phere at most institutions must be judged quite aut or ^ 

ticularly as directed toward student. In the few co h^ges in^the 
Study where greater attention is paid to studen op 

students are allowed a responsible voice in t larger 

tution, gains on tests revealing democratic 

than elsewhere. While considerably more res oarticipate 

reasonable to suggest that example and °PP°'^“"' J j[,ose tradi- 
">ay be better techniques for changing attitudes than those tra 
tionally associated with collegiate instruction. 
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IMPLICATIONS AND UNRESOLVED 
ISSUES 


I N PREVIOUS chapters the activities of the Cooperative Study of 
Evaluation in General Education have been reported in a 
reasonably objective manner, although the commentary and 
the interpretations must inevitably reflect something of the pre 
conceptions of the writers In this chapter, we shall attempt to sum 
manze the more obvious implications of tlie Study and to point 
to several hypotheses needing further investigation We shall also 
attempt to make a personal and subjective appraisal of the Study 
an of general education as viewed through the Study by the 
justification for the latter is needed, it lies primarily in 
e 0 igation which we-as participants in all committee and 
or s op sessions, as visitors to the various cooperating colleges, 
th “”®ultants at other colleges interested in the activities of 
^ organize our experiences in the various phases 

le tu y and relate them to the total general education move 
ment ° 

Although the Study developed out o£ a concern for the collec 
narfir effectiveness of general education or, more 

Its types of general education in achieving 

HIrp^^r, accepted goals, the unique freedom allowed the 

emnhas?s ^ ^ ^utercollege committees resulted in a shift m 
cordpH f/h eommittees interpretation of their functions ac 
mevitnhlu ^ instruction as important a role as was 

and tn tJi the development of evaluation instruments 

T opemtions of the 

.cant cate studT.n ^he^fn/S' ^ 

ratinnai met » » ®^lue o£ Cooperative endeavor among edu 
cational institutions Thne . 

inus, the complex nature of the project 

242 
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results in a variety of implications and in a situation in which no 
one but the directors have quite the overview necessary for an 
appraisal of the Study. This is a further justification for the sub- 
jective nature o£ sections o£ this chapter. 

The views or opinions expressed in this chapter are the responsi- 
bility of the directors alone and are not to be imputed as views 
shared by the intercollege members, the cooperating colleges, the 
Committee on Measurement and Evaluation, or its parent body, 
the American Council on Education. This does not mean ^ at 
originality is claimed for these views, but rather re-emp asizes 
the personal nature of this particular statement of those views, 
many of which are not firmly based on objective evidence. 


The Four Areas of Contribution of the Study 
The achievements and implications of the Study may be ex 
amined under four major headings. First and most tangi e are 
tests or evaluation instruments and the instructiona materia s 
veloped. Ignoring preliminary forms or discarded instrumeri , 
eighteen different tests and forms of tests were developed. jU- 
though some of these, particularly the instruments dealing with 
critical analysis and judgment in the humanities, ave ‘ 
tional implications, the three major contributions to ™ ,j,pme 
techniques are to be found in the handbooks deve ope r 
analysis, for teaching critical thinking in f 

teaching science reasoning and understanding. o j. . 
information about their availability, are listed in ppen ix . 
additional handbook for the teaching of critical 
tuent in the humanities has been outlined and pro a y 
become available at some later date. 

The second area of achievement is in the co ection o 
gains made by students as determined by pre- an P?* . , , 

These gains have been discussed only in reference to t e “ . . 

tests. Institutional patterns of gain and attempts to 
these patterns provide an even more interesting an c 
result of the orotert • ma 

A third contribution growing directly out ^g”°”*i,°ich 

« to be found in the formulation of several 
merit further investigation. These involve the nature o gra 
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tion, a possible way of measuring it, and a series of suggestions as to 
some of the factors involved in gains. 

The fourth contribution of the Study is to be found in the 
extent to which experience in it promoted changes in the par- 
ticipants and, through them, changes in the various cooperating 
colleges. 

Since all the tests and instructional materials have been ade- 
quately discussed in preceding chapters, only the contributions 
in the second, third, and fourth areas will be discussed here. 


Re-examining the Test Results 
The statistical evidence on gains already presented suggests that 
(1) students do make some progress in regard to general education 
goals; (2) the amount of progress is apparently dependent on the 
nature of the educational experience provided and varies in ways 
which correspond reasonably well to identifiable elements in that 
experience: and (3) general education outcomes measured by the 
Study instruments are interrelated, and the degree of interrelation 
varies among the colleges. These generalizations are based on pre- 
an post-testing of freshmen in the cooperating colleges, the data 
being too limited for any significant analysis. 

^ e variation in the size of the one-year gains among colleges 
raises the question as to whether there may be identifiable ele- 
ments accounting for these differences. The most obvious possibility 
IS at the objectives measured may receive more attention in some 
courses or colleges than in others. There is reason to believe that 
largely a matter of maturation rather 
nf ° ^ general education experience. In fact, in many 

tiifi ® colleges gains in intelligence or in college apti- 

fevl^r a one-year period which compare 

favorably with the gains made in the kudy tests. 

terest tn ™ight into the variation in gains, it is of in- 

eains on ah 7 " “’■‘am schools consistently register the larger 

of instituti '>'■ whether there is a tendency for different types 
tests I a w' “ ‘he various 

then the rh ‘™‘l ‘o make the largest gains on all tests, 

stmL "' o„n”maK“'“c‘’' as cLpared with a 

attemot to pv l ’ smallest gains become of interest in an 

attempt to explain their apparent effectiveness. 
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There are, of course, many reasons why the statistical results of 
the Study need to be interpreted with great caution The data on 
gains from the various colleges are incomplete, partly because of 
the difficulties in obtaining post test data, partly because some of 
the test forms used— for example, Torm B of the Test of Critical 
Thinking-turned out to be unsatisfactory, and partly because 
some of the committee members had little interest m the collection 
of statistical data The length of the Study forced pre- and post 
testing a year earlier than was desirable, with the result that some 
aspects of the testing were not well coordinated because the com 


Ranks op Fotm 


TABLE 60 

Colleges in Gains Made on Evaluation 
Study Tests over One Year 



mittees had not been brought to the stage of thinking 
such matters Another difficulty in interpreting test results is tn 
lack of adequate objective indication of the characteristics o p 
ticular programs Visits on the various “mpuses, and the c os 
contact with committee members during the peno o e 
increased the bases for understanding of programs bu e 


understanding heavily subjective . 

In compiling the gams made by colleges (making a 
ffie ability level of the students involved), four institu lo 
found consistently to rank one, two, or three in t ^ , 

unde For example, using only results from a sing e an 
used form of each test. Table 59 shows the ran o 
five different tests Other colleges involved in each tabulation hju 
been dropped in order to emphasize this pu“'uular c common 
These four colleges superficially, at least, have i 
One IS a large publicly supported -te .nstitutmm tlu^^^^ 
privately supported liberal arts colleges T Three 

one IS a college for women, and the other a college me 
are selective to various degrees on admissions, the s 
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requires standing in the upper 50 percent of the high school gradu 
ating class It would be easy to elaborate on differences among 
these four institutions, but the really crucial question is whether 
they have anything in common which differentiates them from 
another group of four colleges registering low gains on the tests 
Four such loiv gain institutions were found, and at this point ivre 
shall attempt to set up in double columns the contrasting charac 
teristics of the two groups of colleges These characteristics are 
the more obvious ones known to us There may be others more 
subtle which have escaped our observation 


Charaeterislict of 
Inslilultons mth Large Gains 
1 Most students reside on the cam 
pus 


Charactensttes of 
Institutions with Small Cains 

1 Large percentage of students com 
mute 


2 Admission involves more than high 
school graduation 

3 A core program o! general educa 
tion required of all students 


High school graduation sufficient 
for admission 

No set program required of all 
students possibly one or uu? com 
mon courses 


4 Program of exemption or aedit by 
examinations 

5 Staff of general education courses 
strongly general education on 
ented 


6 Broadly conceived evaluauon ac 
tivity a continuing part of the pro 
gram ^ 


7 


General education 
confined to courses 


experiences not 


8 General education 


counes not 
r major m related field 


9 


Strong administrative 
active coordination of 
catmn as a unit 


interest and 
general edu 


4 No exemption or credit progr^ 
by examinations 

5 Stall of general education courses 
oriented to specialized offerings 
and teach a general education 
course as a sideline 

6 Little evaluation other than regu 
lar examinations made by indivi 
ual teachers 

7 General education 

largely limited to that provided J 
general education courses 

8 General education courses com 
monly not taken by one majoring 
in related field 

9 General education is provided 
through largely independent u 
coordinated courses 


evprv^^^ contrasting characteristics are not always characteristic o 
four ° g^^^ colleges The first 

judgment, are associated with every 
on of the respective groups The fifth characteristic m 
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volving the orientation oE stafE is a matter 

low-gaL gronp has a staff strongly dedicated to 

butle bSieve the staffs in the remaimng f of 

tions are not general education onente . i ^ exception to 

the low-gain group there would ^ ® Hgh-gain group, 

each oE the characteristics 6. 7, 8, and 9. I S 

there is one exception to characteristic arts 

tion in the applicability of chamcteris .^ educational 

type of private college accords ^re very close and by 

planning, and student contacts wi , JJte school has 

no means limited to the classroom. The .e- 

extensive extra services for individualizatio .Uro„„h formal 

complish most of its general education objectives 

instruction. Perhaps the results of "tuation are 

Table 59, which shows the and Critical Think- 

colleges with highly integrated j caution. 

The preceding characterizations must be aunears 

but to the exten? that our data and j“dg™ents are val d 
that general education, if successful m its 

left to chance either in the courses o ere , g„t;rely from 

them, nor can it be regarded as something achievable entirely from 

a nrogram of formal instruction. 

otheses for Further Research 
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requires standing m the upper 50 percent of the high school gradu 
ating class It would be easy to elaborate on differences among 
these four institutions, but the really crucial question is whether 
they have anything in common which differentiates them from 
another group of four colleges registering low gains on the tests 
Four such low gam institutions were found, and at this point we 
shall attempt to set up in double columns the contrasting charac 
tenstics of the two groups of colleges These characteristics are 
the more obvious ones known to us There may be others more 
subtle which have escaped our observation 


CharaetensUes of 
Institutiom with Large Gairu 
1 Most students reside on the cam 
pus 


CharacterisUes of 
Institutiom with Small Gains 
1 Large percentage of students com 
mute 


2 Admission involves more than high 
school graduation 


2 High school graduation sufficient 
for admission 


S A core program of general educa 
tion required of all students 

4 Program of exemption or credit by 

examinations ^ 

5 Staff of general education courses 
strongly general education on 
emed 


6 Broadly conceived evaluation ac 

tivity a continuing part of the pro 
gram ^ 

7 General education experiences not 
confined to courses 


8 General education 
waived for 


courses not 
major in related field 


9 Strong admimstratiye interest and 
active coordination ot general edu 
cation as a unit 


3 No set program required of all 
students possibly one or two com 
mon courses 

4 No exemption or credit program 
by examinations 

5 Staff of general education courses 
oriented to specialized offerings 
and teach a general education 
course as a sideline 

6 Little evaluation other than regu 
lar examinations made by mdivid 
ual teachers 

7 General education experience 
largely limited to that provided by 
general education courses 

8 General education courses com 
monly not taken by one majoring 
in related field 

9 General education is provided 
through largely independent un 
coordinated courses 


charactenstics are not always characteristic ot 
frti 1 ,”^ o t e our low and four high gam colleges The first 
four characteristics, in our judgment, are associated with every 
stitution of the respective groups The fifth characteristic m 
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portions depending on the particular college, course, instructor, 
student, or test. 

Another difficulty is that any of three hypotheses as to the rela- 
tion of gains and initial status is plausible. It might be argued with 
some reason that general education is a leveling agent so that the 
differences between students should decrease. In this case the small- 
er gain for the initially more able student is to be expected. It 
might be argued that all students should improve equally if the 
program is properly adjusted to individual differences, in which 
case gains should be equal over the entire range of pre-tested ability. 
Finally, some teachers might take the view that the best-prepared 
student has the greatest possibility for further development and 
should make a greater gain than the less well-prepared student. 
Evidence accumulated here seems to favor the first of these three 
hypotheses, but it is quite probable that careful planriing of in 
struction and selection of appropriate test materials might resu t 
in evidence favoring either of the other two hypotheses. This is one 
of the unsolved— perhaps unsolvable— problems raised in the stu y. 

Regarding intercorrelations and integration.— Interconelations 
among the various tests of the Study have been exhibited in earlier 
chapters. From these it is evident, as was expected, that the ''arious 
evaluation devices of the Study are mildly intercorrelated and they 
are likewise correlated with other tests in about the manner to e 
expected. The hypothesis concerning correlations which we were 
most interested in investigating was that intercorrelations among 
tests involving general education objectives should increase 
result of a significant program of general , 

The rationale for this is simply an extension into the sta i 
area of the concept of an integrated personality as one ° 
developed all his abilities and who can and does ring to 
his resources upon any task confronting him. If this means, 
seems to, that such an individual is able to think cntica y ' 
gard to a variety of problems and has overcome or at eas 
of hU bias and prejudice, then a program contributing 
of development should result in an individual who perf 
consistently over a wide range of tasks. It follows t la 
P’tng these tasks and abilities should be more highly 
*an before. 
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found even with certain tests for which even the highest pre test 
score was far from a perfect performance Even so, something of 
these effects must be present 

2 A regression effect, or tendency on retesting to move toward 
the population mean On a chance basis there is more possibility 
for low scores to be higher than to go lower and for high scores to 
drop rather than to gain Unquestionably this factor is involved 

3 A focus of instruction effect, that is, that instruction may 
be aimed at the average or below average student rather than at 
the able individual One study in the science area in which the 
largest gain found was for a special section composed of the most 
able students is consistent with this possibility Comments from 
stu ents interviewed on a number of campuses and classroom ob 
sedations also give credence to this as a contributing factor 

A linfamtltartty with terminology and concepts 

u ents with low scores may be unfamiliar with the vocabulary 
or concepts used m a test even though a definite attempt was made 
factor The gain for low pre test students may 
knowledge than in the ability supposedly 
his mm -.t ^ ^ pre test student must make 

structinn ability itself Content oriented in 

struction ivould then favor the low pre test group 

atina in motivation This factor is conceived of as oper 

what morp rn different ways Able students, being some 

beginning of themselves when tested at the 

soDhistinipri r.» L ** grading, the more 

to spend much uw7on27^ seriously and refuse 

scr\e to inrrp-* u morc difficult questions which would 

-"n ^^hrseen :r:: 

cnees since some student magnify institutional differ 
preciatne of ntt^m groups are more habituated to and ap- 

T.?' e ri:er3a:: r 'I' 

be still others not yet enviLTr 

attempt to explain thr - « ® ° ” apparent, however, that any 

ond g!:,ns IS fnnght ^"rd r 

imoKed, but esm then thJ!"' “'I 

they may be involved in different pro- 
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Table 60 exhibits a pattern of pre- and post-test intercorrelations 
for two colleges. One of these is College A of the high-gain group 
and College E is one of the four in the low-gain group discussed 
earlier. 

For College A the intercorrelations increased from pre-test to 
post-test in all nine pairs and for College E in eight out of the nine 
pairs. The exception for College E involves only a slight decrease. 
The magnitude of the increments in College A is considerably 
greater than for College E. Tests of the significance of the differ- 
ence of the pairs of correlations have been applied, and many pairs 
are found not to differ at a statistically significant level. Since sta- 
tistical significance depends on the number of cases, smaller i 
ferences for College E may be significant. However, si^i cance 
here does not depend on individual pairs but on the consistency o 
the pattern, for a random relationship would result in some in 
creases and some decreases in correlation. The two institutions 
were the only ones for which post-test intercorrelations were con- 
sistently higher. On the assumption that gains or losses are equally 
likely, there are about two chances in 1 ,000 that a ^in wou occu 
in all nine pairs of correlations. The chance of gains in a ut on 
case is about two in 100. On these grounds the results here ar 
significant and in accord with our hypothesis that intercorre 
should increase as a result of general education, ^nrt ermore, 
increases in correlations for Institution A, already s ovm m 
^srge gains, are found to be larger than for College E, w ic re 
tcrcd small gains on the tests. These results correspond to s 
jective estimates of the extent to which the educationa 
at the two institutions affords opportunity for integration a 
the apparent fact that the amount of integration possi e 
Sram must depend on the extent of changes in the stu en 

In Table 61 is shown another comparison of pre ^ 
correlations. As indicated by the letters, the college i 
A and C of Table 59 in which it ivas indicated that Coil S ™ ^ 
'he largest gains recorded on several of the tests, me 
not use the Problems in Human Relations Tc , 

I'ons compared are fewer in number tlian before. 

College A shows an increment from pre- to post tes _ jj ’ q 
Hie data here being identical with those given earlier. E 
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possibility that the magnitude o£ test mtercorrelations is influenced 
by the school s program There is much room for experimentation 
on this matter Meanwhile our results permit a suggestion that the 
availability of uncoordinated courses taught as an extra respon 
sibihty by a group of part time specialists is not the route to the 
best results on general education objectives There is also an in 
dication that although a group of well organized but mdepen ent 
courses taught by a full time faculty may produce excellent results 
on cognitive outcomes, it may not do much in producing eve op 
ment of students in the affective area The fact that the our co 
leges making the largest gains have shown concern for improve 
testing and comprehensive evaluation procedures is also suggM ive 
Curriculum construction for general education, if our evi enc 
and our hypothesis based upon it have any validity, is tttuc 
than the matter of asstgning separate groups of teachers to eve op 
number of discrete courses 


Effect on Participating Individuals and Colleges 

Opinions differ as to the extent to which a 
tivc educational project should be justified on the basis o i 
on the individuals and colleges directly involve lem , , 

«nl desire on the part of those who support 
affect in some significant way the total pattern o eve op 
education by producing results which cannot be ignore 
teahstic conception of the nature of educational researc 
potential consumers— the teachers— points to t e possi 
ony research evidence will be so qualified by the varia 
elements of teachers and students that the imme i 

research results will be slight Belief in the resM« and 
eeptance and basing of practice on them by 
eolleges will be more effective than wide f ^^emmation^or 
findings The prestige of personalities and of O 

tesearch data-'imtating as this may be to ' ’e /e^e-i ;er ^ 
fiasis It IS appropriate to examine the effect of t le 
on the participants mdisiduals It 

I'lttirall), the changes were most marke ,,^1, o[,l,e 

I'ould be possible to point to three or four ,1,5 Stud) 

eominittces isho des eloped through their cxperie 
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TABLE 61 

CoUPAIUSON or pRE- AND PoST TeST CoBRELATIONB ON EvALTTATIOW 
Study Tests fob Two Colleges 


CoixuB 

CmncAt Thitoko nt 
&OCUL Science wrb 

ScrescB REAsoMQia & 

1 llHOBBSTANDINa VITH 

Cbitical 

TniNEiNa 

No ON 
IVhicb 
C oBBEU 
nONS ABB 

Based 

Cr t cal I 
Tlimk aj 1 

1 Inveatonr of 

1 Bebela | 

1 Cr t cal ! 
ThinL 02 i 

1 Inventory of 
Bel efi 

Inventory of 
Bel efa 

Pre 

Post 

Pk 

Poat 

Pie 

Poft 

Pre i 

Foil 

Pre 

Poet 

A 

c 1 

CC 

63 

67 

55 

46 

29 

63 

24 

56 

58 

70 

41 

29 

19 

61 

27 

42 

21 

43 

17 

45- 88 
238-447 


shows a decrease in all but one of the pairs The general trend 
clearly suggests that integration to the extent that it is defined by 
our correlation hypothesis, is not obtainable through good courses 
unless definite steps are taken to interrelate them in or out of the 
classroom In College A each student is a member of the coromun 
ity government made up of students, faculty, and members of the 
CO cge administration, which is responsible for policy decisions in 
both the administration of the college itself and in campus com 
unity uing There is no phase of the college operations— faculty 
planning college maintenance, or planning 
in ^ ^ 'vhich students do not have a direct and substantial 

n * u® to this experience can be found in 

ic r* planned general education experience 

pcnden^t courses which are themselves highly inde 

lufrp ^5pect of the contrast between College A and Col 

relation If relevant to the matter of integration is the cor 
correhtinn *"c*ttory of Beliefc with other tests Whereas the 
tinctlv loupr il"’ College C is dis 

Collcue A ft ^"tercorrelations of these other tests, for 

plies that 3tions are of about the same order This im 

coCTitue oiitr ** ^*^ificnl distinction between the affective and 
College A educational experience provided in 

earlier answer to some of the issues raised 

depends unon th pervasiveness of critical thinking 

The cornier educational program Certainly 

pointed out m preceding paragraphs indicate the 
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The full extent of the effectiveness of the Study on individuals 
and colleges is impossible to assess because it is frequently too sut 
tie or too confounded with other happenings even to identify. As 
with general education itself, it may be that there are long-term re- 
sults which are even more significant than the immediate an o - 
vious ones. 


Instructional Deficiencies in General Education 
The average instructor of a general education course has litt 
familiarity with the extensive general education , 

ministrators find money to attend meetings and to 
and periodicals dealing with broad educational ^ . u-, 

faculty member is trained as a specialist and his a lation w 
own professional societies and his own professiona 1°“™^ . 

hooks consume any money he may reasonably be expecte o 
to his professional advancement. This is particu ar y , 

cases where general education teaching is a part-tinm u ’ 
it is still true for the full-time general education staff member who, 
unsure of the future of general education, must eep a . 
and head in the door opening to his 

the high aims and ideals of the general education , 

more apparent in the literature than in the typica g 

cation classroom. . , ram- 

Visits to sixty-four general education classes on s.^^ 

puses during the winter and spring of 1953 p not 

the preceding comment. The following two genera i 
applicable to^ll teachers and classes observed -ouM holdjor 
a clear majority: (1) there is more concern a ou combined 

tribution of a particular course than there is a o (2) 

effects of a set of general orlniring materials 

mtegration is viewed more as a problem .e^,- n of a logical 
drawn from various disciplines through identi c generally 

of principles and concepts than as a charactenstic of a genera y 

educated person. i-,,,mns are actual 

in part the views implied by these genera i ,^,i,ere 

eonvictions of teachers. There are many ;„n point of 

teachers with a broad student-oriented genera nractices hal- 

^■ew simply end it impossible to break away from prac 
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an entirely different viewpoint about the nature and purposes of 
general education and the means o£ accomplishing these purposes 
The changes in viewpoint were evident both to others and to the 
individuals themselves There were several cases in which course 
testing programs were almost completely revised because of the 
interest aroused in improved testing There were several cases in 
which the contacts of several faculty members from a college be 
came the means for developing increased local cooperation be 
tween general education course staffs Individuals in at least three 
of the cooperating colleges found that ideas developed in the Study 
were of interest to teachers and to graduate student groups An ob 
jective survey of the reaction of committee members was made at the 
inal workshop of the Study Dean Horace Morse, who was asso 
ciated with the Study as liaison officer for the General College at 
Minnesota and as consultant to the Social Science Committee, sum 
arize the results ^ There was a nearly unanimous opinion ren 
^ \ ^ Participants that the experience was very interesting 
ating and very valuable to them as individuals They 
having great effect on other 
their own institutions and they wisely assigned 
convevinl^Mi°^^ witmg their final report the responsibility for 
ThlX^ ' “Sn-ficance of .he Study to other colleges 
the effect ^ colleges is less easy to discern than 

niittecs nmn ^ where there were com 

mittees of the^St T ^ whose activities paralleled the com 

the Domt wh “ y* ^ number of individuals were influenced to 
edncitinn H ^ different orientation toward general 

Stud reTa.l""'’'’''’ “Heges the lack of any lelin.ta 

committee pa^uXToTo'’’' 

Tuo \ communicate what they were doing 

nmh.“ s™ ■" ‘“ have rece.ved 

loss from d!e hcuUy™ 

resulted m almost comp 

the other the membersh n wa? 1'°'’“*“ 

of the project ^ P“'^'ly nominal from the beginning 

'Horace T Mone “WTiat r> t» 

Stud>/'“ Journal of ntefter Edur^i jL^’^f*****”^ Think about the ACE Evaluation 

5 December 1953 pp 469-77, 501 



IMPUCATIONS AND UNRESOLVED ISSUES 257 
Student Reactions to General Education 
Interviews with student groups on seven campuses supplemented 
our classroom observations. The technique for selection of the 
students varied, but since we wanted those who had completed 
some work in general education, most were sophomores or above. 
In a few cases our interest in visiting svith some students was simp y 
announced and they came in of their otvn accord. 

In most cases after we made a few brief remarks about general 
education and mentioned the courses involved in the o ^ gen 
eial education program, students indicated by ^eir responsiveness 
an aivareness of the nature of general education an gave some 
evidence that it was a topic of interest in student bu sessions, 
only one of seven campuses was there evidence that stu er^ 
unaware of the existence of a genera! education program w i . 
that case, they saw as only a group of required courses, ^ 
which, however, they could vocalize readily. In ct, it w 
necessary to raise an occasional question with most , 

to direct a lively discussion into the particular c anne ® * 
we were interested. Essentially three questions were raise 
students; 

1. IVTiat values do you see in general education courses? (Are they 

good or bad, and why?) . ™neral education 

2. IVhat differences seem to you to exist beta g 

courses and specialized courses? , ^neral education 

3. I£ you were to advise someone on setung up a gen^l eouca 
program, what three major suggestions wou y 

All but one of fourteen student groups, varying in limited 

to twelve students, unanimously endorsed the i “ ° , 

general education requiremenL Even in the , noted 

5 minority held out against the idea. The pnncipa 
w general education readily suramanze ^ ^ ^oucal 

(!) the general informational and cultural valu , ( ) ^josely 
'■alue in that general education course ten to including 

related to life problems; and (3) the orientationa choice of 

W the opening up of neiv areas of interests (5) 

»aior and volition, (c) provision of a perspective on society 

on one’s chosen vocation. unanimously to 

All groups made these points and all gT 
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lowed by tradition. In fact, instruction aimed at general education 
objectives requires the development of new methods and materials 
and experimentation with them. It is much easier to be critical of 
classroom teaching than it is to provide the means of overcoming 
these criticisms. Yet recurring observations of the type which follow 
suggest that the quality of instruction in general education must 
receive attention. 

^ 1. Students are typically given little opportunity for participa- 
tion in planning or even in discussion. Teachers talk most of the 
time. 

2. Audio-visual aids do not appear to be extensively used. Three 
map references, two cases of passing pictures, three cases of use of 
recordings, three uses of projection techniques, and one use of 
newspapen constituted the complete listing of such aids, other than 
use of the blackboard, as observed in sixty-four classes. 

j as ten minutes was used in some classes for checking 

a en^nce. distribution of papers, and similar clerical details, 
n.* ^ 5ixty-four classes visited only two had an informal and 

arrangement. Others used the traditional chairs 
stools ^rms, arranged in rows, or laboratory tables and 


sentpnr^^'^T'^ commonly do not respond to questions in complete 
or incomplete sentence and individuals 

sentenci'!'h"k'^^^ ^ phrase. Students who begin a 

cut off bv\h apparently will convey the correct answer are often 
tions tearh ^ cr— presumably to save time. Even communica- 
6 r' r "" of this practice, 

would not ^ ^ student question was that time simply 

evcn if,henam.,;a3fane„Lt“ur”“ - — 

>et thcv observ-ations may be regarded as minutiae; 

planned to make the S^neral education classes are not well 
or of student moti\-ado”n* of teaching aids, 

eral education in«t ■ ^^'tcrest. This is not to say that gen- 

hrge, m« the chalI?„geTn ”,ved fa ‘ "‘'"1' v'’ 

b involved in general education objectives. 
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that our interviewees would admit to is that there is such a tend- 
ency at hrst, hut that it can be overcome by careful etylanauon. 
There was further agreement that any required course has to hear 
some resentment and that exceptionaUy good teaching is necessary 

to justify the requirement. . 

A questionnaire study stimulated by a desire to investigat 
further these favorable student reactions was based on a ° 

over twenty objectives, some typically general e ucation in na 
and others typical of specialized courses. Students were as' 
select the objectives of primary importance for attainmen in 
hist two years of college. In the case of every student ^oup 

surveyed and tabulated, the general education objectiv es 

ranked ahead of the specialized objectives. This investiga ’ 
though growing out of the Study, has j 

pendently and will be reported in detail elsewhere. 
added to our summaries above demonstrates cone usiv e ^ . 

student opinion can be cultivated into one ° ‘ ® ^°(.ducation. 
most constructive forces for the development of gene 

The Role of Evaluation 

In chapter 2 the various attitudes of faculty j, 

tion were described, but some additional from 

ing. Even if they accept evaluation as important, i 
other matters equally or possibly more importan . 
teachers-particularly in evidence among y-to certain 

committees, but even to some extent with tna gr attitude 

of the insnaiments developed in the Study , ^^nalysis and 

m another way. Any instrument such as e 

Judgment Test which requires considera ^ practical 

'0 use may be regarded as a curiosity ra er ijQdons and 

■nstructional or evaluatory tool. V^ide from the be 

ihc time element, another objection to su ms rj 5 e of 

found in the uniformity of judgment require 
such devices. • • • ed bv teachers, an 

Although objective tests are complicated 

objeedve test is more likely to be used than 
essay or rating techniques. However, the sesss u defini- 

out sutHcient study or understanding or accep 
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these values Culture apparently is a genuine concern of students 
Many indicated that their general education courses provided a 
basis which they had previously lacked for entering into serious 
conversations with adults 

In every group there were one or more students who claimed 
to have made a complete change of vocational plans as a result 
of one or more general educaaon courses All were able to name 
other students who had had this experience Several students had 
sufficient insight to suggest that the general education courses were 
excellent for students who took only a year or two of college Scat 
tered but reasonably frequent references to such values as learning 
how to think, more effective communication, and the like in 
dicated that a few students had been exposed to general education 
objectives 

their general approbation of general education, students 
1 enti e some differences between general education and other 
courses which are not entirely complimentary to the former 
Among these were remarks that 


and do not challenge 

general education courses are resented by freshmen, who 
typically see no reason for them 

nrn"v^?J classes are usually larger than other classes and 

Genell ? .ndividuabzation 

muclt tokver^*^" courses have less flexibility because there is so 

ones^sTlirt courses are more apt to have poor teachers or 

ones Uho are not interested in the course 

Genwa'} fdur^‘ are heav.ly factual 

eneral education courses overlap many other courses 

recervcd'ltron ' ^ made in all groups and usually 

Comments 5 and majority of the students 
campuses heavily emphasized on four of the seven 

-ipprm edm ’■™”hs all groups-as we have already noted 

which they had ‘he general education courses 

the contention that students ‘’“‘'""e “> “P 

tltat tltej bitterly resent genlr^r T “ particular major 

J g neral education requirements The most 
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such as critical thinking or o£ attitudinal traits. The use o£ the dis- 
cussion method at the College o£ the University o£ Chicago, or of 
the case method at Colgate and Harvard demonstrates that funda- 
mental revisions of instruction are possible, but they are not com 
mon. Even the variations in teacher proficiency and enthusiasm 
raise problems of experimental design which make care u^ ex 
perimentation so difficult that few faculties are apt to engage in it. 
The planning and carrying-out of significant research on t e earn 
ing process require the cooperation of psychologists, eva ua o , 
teachers, and students. It requires time for program rep ann o> 
for the development of additional evaluatory techniques, 
tensive testing, observation, and follosv-up of students, nap 
of increasing enrollments it is improbable that much exper 

tion will he done. Nevertheless, the final chapter of . 

he devoted to the presentation of a rationale and a roug in ica 
of one direction which such experimentation might ta -e. 

Lacking such experimentation, there is still muc ^ „ 

done to improve general education. First of al , e ^^^-n-rhes 
of colleges which have already embarked upon nove ®PP _ 
tan be studied and drawn upon by others. ^ v . 

fainly has possibilities for use in many courses, ew -oger- 
"■ant to adopt it to the exclusion of all other pe 

ate usage and even a subjective appraisal o . , , relevance 

beneficial. Second, the introduction of the pnnciple 
^ a basis for classroom planning can be use . a” mpan that 
tritical thinking is accepted and if it is interprete -ijssroom, 
this ability is to carry over to problems faced outsi e , rning 
it is not unreasonable to expect that sheer J 

ni content might be somewhat reduced to P™y' ^ student, 
nppottunities for critical thinking on the P^f nrovided bear 
biorcover, it is reasonable that die °PP°’^‘""1 “Jp-v is usable 
some relation to tlie situations in which this 
onisidc the classroom. This is simply an apphea 
ople of relevance. From one point of vicsv, t ii determined 

use of the case method. Rcle\'ancc wn r jtudent 

arc operationally define . j jnust be 

^'mvior. Once again, evaluationxentered *>"'-'"5 

®'ntioncd as a significant approach to die problem. 
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tions and objectives involved, so that unfavorable results are used 
as a basis for discarding the test rather than as a basis for re-exami- 
nation of educational practices. 

Effective use of a completed test requires either construction 
by the instructors using it or sufficient study of it by the users so 
that the purpose of the test is fully understood. Successful intro- 
duction of evaluative activity into a general education program 
involves both the development of interest in such practices and of 
^rae proficiency in making and criticizing evaluation instruments. 

e types of activities in which committee members of the Co- 
operative Evaluation Study engaged are precisely the sort to be 
encoura^ged among all general education staff members. Almost 
inevita y both evaluation and instruction will thereby be im- 


The Need for Experimentation 
re3nn ^Jicouragement of experimentation was the major 

envisatrpH*- * experimentation originally 

and matpr‘*^i^^ ttt of courses using different methods 

hut on ! experimentation was done 

of certain nW became more fully aware of the meaning 

parity b ^nd were therefore made conscious of a dis- 

onnot simn^r^ existing practices and accepted outcomes. One 
a prornam ^"’™d that a general education faculty embark on 

a Ss seen'^rr'?'""- waning until 

which lead-! i -c activity on the part of the faculty 

■ntns o t n. -"'1 development of 

instruction and'"l''“ attainment suggests deficiencies in 

''•Wchcanbecom?tLXto7“““'^ 

Even at this point mu* If 

be without si^tificance Gen^VT™"'""™ “"P' 1° 
cover so much material .h., ‘ courses commonly 

superimposed on an airct """ ^ fechniques are simply 

attention is given to Im t’’ 

Only .as courses and ( ° P°“‘EIe significant comparisons, 

compared with ‘ P^Srams are completely revised and 

to give t^ to^ZSer 

J fferences in the development of abilities 
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A Re-examination or tiie Role of Objectives 
In General Education 

General education has been characterized by concern abo 
objectives; yet months and even years of effort by acu ty co 
mittees and by total faculties have been expended on o ^ 
with little overt effect in the program. Tliere are many re 

why this is true: _ , 

1. The effort o£ harmonizing philosophical differences ^ ' 

coming semantic blocks commonly makes for su cien i 

that the achievement of agreement on objectives may e 
task complete in itself rather than as an initia . jjj 

plete step in the development of a curriculum. ;(:„„iiat 

effect in such cases is that each teacher reinterprets or ju 
he is doing— and continues to do— in the light o t | 

2. The tendency of administrators or faculty , 

write objectives primarily for catalogues without _ 

and with little attention to whether faculty know, ^ 

thing about the objectives. The remark of ® illustrates 

partment head when asked about his cour j 

the point. His reply was to the effect that he believed that the 
had a'set in his file but no one paid any attention to ^ 

!. Tl. r,U., *1""'“ » «« 

organization of the Study oommittees-deleaU 
integrating force supposedly inherent in o j 
them appear to be descriptions of the ' i,e largely 

grative type of objective appHrahle to a 
Ignored by all. General education courses 
in overcoming the fragmentation of knowledge nut 
about overcoming the fragmentation of j their gen- 

The number of objectives common V ® single teacher 

crality encourage teachers to take the ''io"' ® result is reflected 

Or course can contribute to all objectives. ^ j.,j to the intro- 
■n the remark of a communications teacher in r j-lassroom. It 
ilnction of some group dynamics techniques i resulted in 

Was to the effect that this could be justified o y ^^.^ieved by 
Poater improvement in speaking and writing 
his current methods. 
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The Need for Cooperative Effort 
Teachers are busy people and general education teachers are 
busier than the average The production of syllabi embracing 
broad areas, not all of which are familiar to the teacher, has com 
monly been an assignment added to a full teaching load Release 
of time for planning, for evaluation, or experimentation is rare if 
not unheard of Even if some slight time release is given, manifold 
routine duties have a way of infringing upon it so that little is 
accomplished 

Another difficulty in general education planning is that even 
a new course rather quickly crystallizes New ideas proposed by 
an individual to his own associates may be greeted with in 
1 erence if not derision Even administrators hesitate to suggest 
that rethinking of a course is necessary 

sn™?f^u opening of college have been 

runnnn^ ^ colleges to provide a time free of inter 

studipH p”"® which curricular revisions and other matters can be 
offers activity among a group of colleges also 

colleires Association with individuals from other 

of ideas fh ^ common problems result in a wider range 

spom b o" one campus A d.v.s.on of re 

u uro ri ' ^"'ons colleges may make for more progress and 
Me^m^s” oach person to fulfill his assignment 

ce3 ■n'occollege groups ensure a penod free for con 
the invesim ^ ^ problem Selection for the assignment and 

on those tonds tn such activity confer a prestige 

Is^ered ‘ ''‘“Ity thL such activity is 

administration 

CooDcrari\*f» Scogiaphically limited versions of thf 

well be orcani?'',!'^ Evaluation m General Education could 
and progrcM on'^throrohr'"' a““ring continuing attention to 
the authors has nr/ attacked by the Study An article by 


presented the arguments for such regional co* 


the authors has 

operatise projects' m giaatcr detail”- 

* Paul L. Drcjsel and Lewis R \f, ». 

llonal Planning and Research " Cooperation among Colleges In Edua 

t-aurattonal Hecord. Apnl 1953 pp 121-31 
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For example, the concern of the Science Committee with the 
reading of current scientific articles and the associated abilities 
involving scientific method and attitude may obviously be applied 
to materials dealing with health, selection of a mate, child care, and 
the like. Such materials were represented in the original collection 
from which the test selections were draivn, and their absence or 
near absence from the final product simply reflects lack o ^^^ttip asis 
on such matter in current science courses. Again, the conten 
axis of the Critical Thinking Committee includes problems invo v- 
ing self, others, society, values, and ethical standards. f” 

vocational choice problems, among others, were consi _ 

entirely appropriate situations for the application o ._ 

ing. Certainly the problems and statements employe y 
tudes Committee impinged upon all areas of living. . 
behaviors and the range of situations of the objectives “ 
vestigated, included or might readily have inclu e 
is involved in objectives dealing with home and ami y i 
choice of a vocation. 

One reason for this inclusiveness is that some 
objectives like critical thinking involve a W® ° J aoplica- 

others define a class of situations wherein this e a , 

ble. A more important reason is that the objectives o g 
cation are not discrete even though they * , ,, sjgnifi- 

evaluator or teacher who seeks earnestly to n gf an 

“nee of any objective as it assumes meaning in ;_,er,voven 
individual will find all other objectives inextric ^ 

'rith it. In a sense, our concern with many ° living 

Vays either an unwillingness to view education as, 

1- Select a limited number— four or five at mo of knowl- 
''ithout concern as to whether these cover every 
"ige, every ability, or every area of living- oiade to each 

2. Ask each instructor to describe the con ri . techniques 

“i these objectives in relation to ^ ^ . .odents. 

“ted, and experiences available to or require 
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5 The tendency to separate the statement of objectives from the 

delineation of the educational experiences necessary to evoke and 
give practice in the behaviors implied by these objectives makes 
statements of objecuves an academic exercise 
With the completion of a statement of objectives, the process of 
urse construction proceeds in the time honored manner by 
ipt?,/"i T “urses and by selection of appropriate ma 

mat ^ “idplexity and the number of the objectives 

wSn P”«'‘:able one, particularly 

Imter?L ““y °£ the objectives 

Sion obSr “ development of a list of general edu 

m develoiinr*^^ ’’ eommonly a semantic and logical task helpful 
verbally fxnf “mmon vocabulary and a superficial unity in 
Sect on XS ' having little 

lem in our final*° i, ° Pt°P°te one solution to this prob 

exSSnce of th doing so, a review of the 

ing the point of vie ^ to objectives will aid in grasp 

thi prSosaT «P«tence and banc to 

educatio/objeSveSLt'S'f' ““'"’Pt “ deal with all general 

that It IS subject to moment ignore this and assume 

home and family Iivmr'trhf °*’J'=ttives pertaining to 

The validity of such ^ and to selection of a vocation 

solely to the obiertiv^ t:riticism cannot be ascertained by reference 

only after a °^mdToT, >' 

ohjectives ^ ^ mterpretations placed upon these 

Even a study of the inf#»r 

sideration of the list of P^^tations must not stop with a con 
take into consideration ^ behaviors developed but must also 
behaviors are regarded a. ^ of situations in which these 

objectives must be Next the supposedly ignored 

variety of situations in which into behaviors and the 

delineated Comparisons f h behaviors are operative likewise 
become the basis for reach behaviors and situations 

tives have been ignored ^ decision as to whether the objec 
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have little opportunity to contribute to the development o£ the 
general education program. 

4. Because general education objectives are interdependent and 
because some specify behavior while others specify on y c asses 
of problems or situations to which those behaviors are app ica- 
ble, broad interpretation of one or two objectives may actua y 
include all those usually stated. 

5. The objective of critical thinking is one of major concern 
majority of instructors of general education courses. 


The importance and general acceptability o critica 
and the conviction that concentration on one or two p 
objectives by entire general education faculties , . . 

thing approaching an integrated program is to e ac 
us to devote the last chapter to an examination o t e P 
inherent in such an approach. The nature and t ^ ^ 
critical thinking, the development of appropriate 

ness xn terms of student achievement, and th _ uy,t in- 
and consequent motivation of students ® '^Lvplnnment of 
separable set of problems central to the ” attack through 
general education which appear to be aroena 


concentration on this one objective. 
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3 Consider the relevancy and the scope of these learning experi 
ences in reference to the applicability of the behaviors implied by 
the objectives in nonacademic situations 

4 Suggest ways of relating the educational experience of the 
student more directly to the type of situation in which it is ex 
pected that he apply the abilities 

Seek for ways to evaluate the effect of these experiences in 
promoting changes in students 

Certainly objectives and classroom practice must bear some 
relationship to each other, and it is more likely that these can be 
Otil ^ * '™itcd than for a large number of objectives 

struetin “Pl'citly interpreted in terms of in 

ment of on the develop 

ment of a general education program 

Summary 

and wnctCs'' RrexarnLurn"*'/ 

earlier chapters suggests that m Th 

and teacher ® there are institutional, course, 

students and that t 

Sable characteristics of associated with identi 

concept of integration ma '1“’“’ educational experience. ( 2 ) the 

and that these inter 7 ^ associated with intertest correlations 
■ng on th n uro'ret*'”'" 

Observations m clalt,l"““T' 

of Study committee memh interviews, and the opinions 

members led to the con 1 Seneral education faculty 

1 r 1 j ' “"elusions that 

cant role in planning rcours^''‘’“T''^ P''*’' “ 

teaching is highly traditional "" 

2 The multiplicity of 

results in association of o objectives frequently 

course, thus, mdependenr objectives with a specific 

generaJ education think, encouraged and broader 

3 Students are generallv ^ T,'* are discouraged 

plain that it is inadequate^ T "doeation but com 

4 y explained to them and that they 
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effecting a remarkable degree o£ unanimity amongst those who do 
accept it. The contrast between the confusion *at preva b i 
secular institutions and the unanimity of thought in Catho 

colleges has often been noted.’ ' . , 

Items for inclusion in or exclusion from the c^nculum of the 

Catholic college or university can be judged by 

of relevance to the major goal of such institutions. Catholic 

Catholic system is theology; theology in turn 

ethics and Catholic philosophy; and the Catholic P° j. 

the interpretations of history and literature is unmistakable. 

Catholic institutions no further integratiori is nee e . ■] j.jy 

.„d 

well placed with respect to integration, Curricular ma- 

some administrative officials of certain insutution . ^ j 

p„c..ce., -"vr” f b, 

engineering, medical, law, and denta nractitioners-and 

the goal of producing successful professi P 
American society has fairly well-defined . ^j-hools have an 

fields. This is not to suggest that the pro ess medi- 

easy task of curriculum construction. As a ma ,j,gnjselves with 
ral and law schools are currently reappraisi _ ^ 
the thought that their programs have ” Medical schools in 
possible doctors or lawyers for Amencan ne • subsidies, been 
particular have, with the assistance of more general 

investigating the possibilities of provi ing a remains that 

kind of medical education.’ However, t e P ^ije various 

as compared with most programs of genera integrated 

forms of specialized education are ^T“'^„ntortunateIy, society 
toward some clearly discernible objective. generally edii- 

has not provided a complete job ‘kosor‘P“°" pm- 

tatcd person in the sense that it has there have not been 

f'ssional persons. This does not mean tha generally cdu- 

attcmpts to make explicit the characteristics ^ py„l,l„g. 

, nattigan. A Critical Study of the , rducalion." 

“J- Caiholk Univenlty of America }®^?mroanitio to ‘I'",'"' .^Uol.ertitr 

llowarU St. jonej. "The Relation “f ■■of ElementJ (Chleaso- 

Omejal Education: It, Nature. Scope and EuenMt 
« Uuago Prni. 1931). pp. 55-Sl. ,,,, 

^«nua/ /import o/ the Commonti’ealth fvn > 
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T he general education movement at its inception derived its 
energy from^ the fact that it was a protest movement against 
many practices in American education. It was a protest 
^ ^ ^'^"^psrtmentalization of knoivledge and the prolifera- 
associated evil of the free elective system. It 
as esqpmi n” the Germanic conception of a university 

encv of Am ^ ^ ^^^^^ch-oriented institution. It decried the tend- 
and loffir ^ scholars to be more concerned with the content 
individuals P than with undergraduate students as 

" hat ^ ^^bject during the past 

namism of nro7 J exploited these protest possibilities. Dy- 

replaced by a less n^Mlisd' ** 

pendent on a source of energy, a movement de- 

a significant f “ ’'“und to falter and to cease being 

been as impressed with the 

ministrators and nth 'validity of the protests as have ad- 

lems. The general and speakers on educational prob- 

more positive dynaL^r n " - “ presently in need of a 

the principle of protest. tntegrating principle to replace 

vith respect to this^*^^^^*^°*^ been more fortunately placed 


matter of integration. Catholic colleges and 


universities have generalKr •« » ” ” 

here been conceived b accepted general education as it has 
was already well inteiJT'a' feeling that their education 

This is succinctly exnrLLnT“n'* point of view. 

non-Catholic educator^mii.t •^“igan. "However much the 
agree at least that the anth '?®^';"'“b this viewpoint, he must 
onty of the approach is capable of 
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Good Citizenship 

Some individuals have argued that general eduration courses 
ought to be oriented toward some more pragmatic value than 
knowledge. One fairly popular principle has been that the ine- 

gating factor should be the goal of effective citizens ip in mer 

democracy. If this goal be construed broadly, there can be no 

quarrel with it. Unfortunately, when attempts have 

define this objective operationally, they have eit er ^ 

pletely because of the disagreement or dissatis action 

else they tend to resemble the indoctrination foun in , 

states. In addition, teaching for citizenship in raerica 

racy is too frequently a provincial concept resting up 

sumption of the superiority of the cultural heritage 

civilization. 

MuULife . 

A closely related integrating principle an as- 

of preparation for adult life. , ,„d future needs 

sumption that teachers can identify the pre needs, 

of each student and can select courses con- 

A^in the principle is a good one, Sunura jo be 

reived often includes such a variety of cou replace. This 

guilty of the weaknesses of that which it see * ^ quality of 

conception may also lead to the altering o become mere 

general education to the extent that and vague- 

rocational and avocational education. T j-jg coupled with 

uess of citizenship and of preparation for a “ [g accept 

*e reluctance of many believers in ugation for 

mem as other than ultimate goals with itt e j.pptable as in- 

wrriculum building, make these objectives unaccep 
legrating principles. 


i^olues 
There 


, . . , ,bat religious or humani- 
ze are others who would insist general educa- 

”ian values should be the basis for planning ^ some- 

“un curriculum. This would bring to jed. Observa- 

mmc of ,v,o : C .T,„ /-nfUnlic position eariie_ „„ tbc 


u curriculum. This would bring to Observa- 

®g of the unity of the Catholic position e ^gjpmittee on the 
‘mn of the difficulties of the Cooperative Study 
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Gated persons Many views are argued with great vehemence, but 
none has satisfied all or even a majority of people concerned with 
general education 


The Inadequacies of Various Integrating Concepts 
Common Knowledge 

One view originates in a concern that general education pro 
vide a background o£ common knowledge Certainly general edu 
cation courses heavily emphasize knowledge Teachers do com 
monly admit that it is not possible to determine ]ust what knowl 
e ge the student should possess but having selected certain ma 
term s and specified certain knowledge, there is the tendency to 
emphasize these elements to the exclusion of all else One might 
accept t at one institution finds it imperative to require a course 
soil', I*'!? ''hde another requires biological science, and 

mril ° ” bodi, if the materials were primarily or even 

fran "“‘"y of ceruin abilities and understandings 

evidence However, there is little 

mind the t ^ 1 ^^ transcendental objectives are prominently in 
the oartici.Ur ' implying the importance of knowing 

covered and th ' ™vcred in class Because so many ideas are 
such courses imporunce is not accepted by everyone, 

ality As the troublesome charge of superfici 

adequtly ev^b?' 

sketchv Thp .ni ♦ ^ sampling must become more and more 
adaptation of a block and frequently been some 

information are taueht "'’“troby certain segments of 

knoi, ledge left untouched OKe u' 

and tlie gaps excluded he u particular blocks taught 

of teachers holds the bala^ '’"t" ‘^'radod upon by whatever group 
cation coune That such de“ ® Pttftttttilar general edu 

That they preclude tre „r77^" 

truly general should be equaIIv!o’'w education 

the size of tlie blocks ,nc? j '/ Worse, whatever the gaps left. 

surrey courses Emphasis on'an7'‘/““’‘' 
knoiiledge is not an adequate blrtoT" h 

education ^ further progress in general 
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Good Citizemhtp , education 

Some individuals have argued that ge valu^ t 

ought to be oriented toward some mo t e 

inowledge One fairly popular citweuship „o 

gatmgfactorshouldbethegoaloie umadly. ^ ode to 

^ li g,., V 

quarrel with it UnEortunately, "whe either f 

define this objective operationally* ordissat.sfact>°"^^°^l.tarian 

pletely because oE the disagreem found i (jetnoc 

else they tend to resemble the indoc ,n Anioi^^^ 

states In addition, teaching Eo^ nncept resting Western 

racy is too frequently a provincia heritage 

sumption oE the superiority of t lo 

civilization 

rne is that 

'fdull Life sugg®*'-®'^ on an 

A closely related integrating pt>" ^pceptm” ^ , future 
Of preparauen tor adult ffo these needs 

sumption that teachers can ^„,ses to'"''"" aucatm" oo^^ 
of each student and can sele general os 

Again the principle is a goo „ of oont leplao® 
oeived often includes such ^ ,,hich 't -ssenua' * jg 
guilty of the weaknesses of * of the^^^ '’"""'^^vague 

oonception may also lead 0° ^jiat »" 'J'^^eomplo’"'''' ,^,th 

general education to the e j,on , yt liEe accept 

vocational and avocational e \ educatr®^ £ ^ 

ness of cimenship and of prepa- ,„.p,.cat.on 

the reluctance of many h ^vjth h ^naccep 

them as other than ultimate ohje^t* 

eumculum building, tn^ 

tegrating principles humani 

♦uat rchgr°" 

HO ‘r“olanni"8 avocation some 

There are others w for F gcucml e Ohserva 

tanan values should ke tl;^„ Hr.ng^- « oarl.c^-"f„„ „„ .He 

lion curriculum P^.,^c Study 

thing oE the unity oi Co®P^ 

tion oE the difficulties o 
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cated persons Many views are argued with great vehemence, but 
none has satisfied all or even a majority of people concerned with 
general education 

The Inadequacies of Various Integrating Concepts 
Common Knowledge 

One view originates in a concern that general education pro 
vide a background of common knowledge Certainly general edu 
cation courses heavily emphasize knowledge Teachers do com 
monly admit that it is not possible to determine just what knowl 
edge the student should possess but having selected certain ma 
ternls and specified certain knowledge, there is the tendency to 
emphasize these elements to the exclusion of all else One might 
accept that one institution finds it imperative to require a course 
m p ysical science, while another requires biological science, and 
s i anot er requires both, if the materials were primarily or even 
fr/ti ^ to the mastery of certain abilities and understandings 
particular materials However, there is little 
^ transcendental objectives are prominently in 

the ^ ® °^tentation implying the importance of knowing 

covered covered in class Because so many ideas are 

such ennr ^ essential importance is not accepted by everyone, 
alitv troublesome charge of superfici 

adeauatelv ^ knowledge increases, attempts to cover it 

sketchv ^^oipling. must become more and more 

adaptation ofl°biocTandt,“ frequently been some 

information are taurhf '"*'^'■*7 certain segments of 

knowledge left untoufhed ‘S"' '^^ge gaps of 

and the gaps excluded h 'I™' Pafr-cular blocks taught 

of teachcrs'^holds the b^lanc^T 
cation course That such de 

Tliat they nrechiHe ti ^t^^icult to defend is obvious 

».i, “‘T ‘'“;r 

education ^ ^ further progress in general 
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science materials,” the shills which seemed to be involved were all 
quite similar. Whether the similarity in conception o£ thought 
is attributable to some basic quality or whether it is merely at 
tributable to the Influence o£ Aristotle, Bacon, and Dewey in 
American schools remains a question. This common concern, ow 
ever, served to emphasize the importance o£ critical thinking an 
suggested that critical thinking, viewed broadly, might provi e t e 
emphasis whereby general education courses, individually, couiu 
be better planned and taught, and whereby general e ucat 
programs might achieve among the courses some l^tpr 
integration. We would suggest, there£ore, that ‘ 

might serve where other principles have £ailed, an we s 
demonstrate the validity o£ this position. 

First we need to clari£y something o£ the meaning ® ^ 
oitical thinking. By critical thinking in this connection w 
mean the limited conception implicit in some o£ t e tes 
eal thinking previously described in this report, or ° . 

“me pat £ormula to be taught at the beginning o ® 
lowed by material presented in the same old content prob- 

Rather, by critical thinking we mean a point o£ 
lems and their solutions and a way o£ thinking a o . pf 

lems faced by mankind. Such a conception o£ cntica J’ 

“ume, implies that it be given the central position in cumcm^^^ 
P anning and not regarded as something ponp 
ttaditional approach. . f,.,a,;ral thinking 

. One reaction to this is that such a conception « ^ntical 
n no more pertinent to general education t an usually 

Perhaps not. It is entirely possible for " 
considered as general education to have “ ® ^^i^ular subject, 
velopment of thinking in the context ° P contribute to 
n courses in technical or professional u„wever, more 

development of the higher mental P™"" ' " to present 
'^R^^ntly than not, such courses or programs technical 

iku™'^ 'vith certain knowledge and to teac limited 

'"s- Any increase in ability to think af merely a con- 

s^ific types of problems or to be regar education 

mitant of the other aspects of the course. , cational goal, 
specify critical thinking as an important educa 
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Evaluation o£ Objectives in the Area of Attitudes, Values, and 
Personal Adjustment, and of a committee composed of represen- 
tatives from the several general education courses at one college 
{which gave up entirely— after two years— on the task of defining 
value outcomes) suggests that any acceptance of common values by 
college faculties is a rather remote possibility and even if achieved 
would be at such a level of generality as to have little significance 
in planning the day-to-day activity of a general education course. 
If the commonly accepted objectives of general education have had 
as little effect on the general education classroom as our experience 
suggests, it seems unlikely that emphasis on the most intangible 
of all those objectives will change that situation. 


Summary 

The above is not intended to imply a lack of validity to the 
wntributions these orientations have made and can make to 
mencan education. There is need for education for citizenship. 
1 ne colleges must continue to produce technical skill for indi- 
1 ua s vocatmns and avocations, and there must be logical rela« 
t onships established among hierarchies of course offerings. There 
^ \ crucial need for colleges to provide some basis for 
° values. These orientations, however, do not appear 
eduntinn^ ^ central principles of organization for general 
rollintr • which are to be taken by all or most students en- 

be None possesses a logic which can 

nrinrinl!. teachers and by all students. Some other 

ecsts the nn the experience of this Study sug- 

gests the possibility of critical thinking 

Thinking as a Possible Principle of Integration 
Importance in General Education 

s;s'r;r ■’“I” 

iiie Six committees defined it, • , . . . e 

criticnl thinking began to ° 

of tvhethet a comtl^ttee laS' Regarfiess 

iudfrmpnt’* •• *.• t . the trait critical analysis and 

judgment or cr.ueai thinking" or "the abiiity to read current 
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With the scientific knowledge increasing beyond the bounds of 
ordinary human comprehension, a significant factual knowledge 
of science from one course is rather hopeless. Such courses can 
develop, by constant attention, some grasp of the methods of science 
and of the limitations of scientific inquiry. Instead of organizing 
a social science course according to logical subject-matter divisions, 
sustained efforts to teach problem-solving might achieve greater 
rewards. Similarly counseling— whether academic, vocational, or 
personal— can facilitate work in classrooms as well as the personal 
adjustment of the students if counselors attempt to develop on the 
part of counselees a rational method for solving problems which is 
at the same time being stressed in academic courses. 

The issue arises as to whether, even if the term critical thinking 
be accepted as useful in all fields, the same phenomenon is meant 
thereby. The nature and generality of critical thinking is still a 
matter for research. Tentatively, however, we hypothesize that 
although some differences exist from one field to another, there are 
many common aspects of critical thinking which apply equally well 
in all areas. Once the semantic barriers are overcome, the methods 
of science may not be appreciably different from the methods in- 
volved in viewing a work of art or in making one. Some analysis 
the problem and a marshaling of evidence about it are necessary. 
Although the conclusions may differ— on the one hand an esti- 
mate of approximate truth, and on the other an esthetic judgment 
of beauty-the processes by which one arrives at these conclusions 
may be similar. Even if they are more distinctive than they appear, 
teal value would be attached to assisting students to see the differ- 
ences. If teachers of humanities and teachers of science would each 
teach for the development of tlie same skills applied to different 
materials, the interdependence of knowledge might be demon 
strated and a richer meaning for each subject implanted in the 

student’s mind. 


and Creative Thinking 

Emphasis on critical thinking might appear to neglect or eve 
antithetical to creative thinking. Knowledge about creative 
‘bought, like knowledge about critical thought, is actually quite 
but we can venture some hypotheses about creative think- 
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and thereby assume a special obligation in regard to the objective 
Other courses which do make overt attempts to develop critical 
thinking skills can and should make a definite contribution to the 
general education of a student Their impact is likely to be en 
hanced if many teachers arc attempting to develop the same skills 
at the same time and with the same rationale. 


Appropriateness in Various Courses 
In the field of communications all aspects— reading, writing, 
listening speaking— involve or should involve thinking and may be 
approached in this way rather than as discrete elements The act of 
imting consisting of formulating a problem, exploring it, and 
reaching a conclusion would thus be viewed as having much in 
common with giving a speech Although the medium is different, 
the problems of choice of words, adapting to the audience, and 
ordering the presentation, are similar 
In the humanities, regardless of the subject matter, the goal 
might be the development of critical facility with respect to hu 
mamstic materials If courses in general education in the humani 
ties are to have lasting value, they ought to provide students with 
the means for enjoying artistic work as adults To do so demands 
criteria for discrimination between the valid and invalid, the 
sentimental and the genuine feeling These criteria may well in 
vo ve t e same method of thought involved in the communications 
processes They consist of determining what ideas the artist is try 
mg to communicate, deciding how he has used his medium, and 
ju gmg t e validity of his conclusions Certainly, from a rich work 
o art sue as Beethoven s Ninth Symphony varied interpretations 
are possi e, even by the same person on different occasions The 
s eps or t e process by which one arrives at these, however, seem 
not to e greatly different Even in the more controversial areas 
re igion an p ilosophy the critical processes by which meaning 
IS o tame rom basic documents are in essence the same as in 
painting, music, or literature— and this does not imply any diminu 
ion o re igious feeling The processes involved when students 
were rc^ir to compare Jesus’ point of view with the hedonistic 
one o mar and the social-criticism orientation of Walden are 
desirable ones to teach. 
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a leaf and told them to observe it, study it, and then describe their 
findings to him. Their first reports were rather prosaic statements 
of the very obvious leaf structure. He then sent the students back 
to look again. Gradually the students came to observe the struc- 
tural beauty, the organic efficiency, and the inherent possibilities 
for further study in that single leaf. At this point and only at this 
point would the teacher be satisfied with the student performance. 
Very obviously the professor had observed all of these things be- 
fore, but he was forcing students to see the same things for the 
first time, and was demanding of them creativity of thought. 

Or, again, creativity might well be encouraged by asking stu- 
dents to consider a view of the Notre Dame de Paris, the Summa 
Theologica, and descriptions of medieval manor and town life. 
Students in arriving at generalizations regarding the medieval 
period might demonstrate considerable creativity or originality, 
even though other people have arrived at similar syntheses. In the 
context of a social science class students might be led from a theoret- 
ical consideration of the dynamics of personality to introspection 
about themselves. Resulting insights are original with the student 
and may exemplify creative thinking. Creative thought viewed this 
way, whether or not it embraces all of creativity, fits quite readily 
into a rationale of critical thinking. 


delation to Other Integrating Concepts 

in addition to providing a rational basis for organizing general 
education courses and programs and to allowing for the develop- 
ment of creativity, critical thinking as an integrating principle may 
prove to be the one most acceptable to many educational points o 

view. 


Whether one believes that the fundamental purpose of a college 
M university is the inculcation of moral and spiritual valua, the 
ttansmission of knowledge, the development of traits o cit 
^fiip, the preparation of individuals for adult life, or t e eve o^ 
uitni of intellectual keenness, critical thinking must p ay a “*1 
r'-’e. To all save the most doctrinaire of 

“ a willingness, even a demand, that rational methods o i q >7 
' used in all Helds of human activity, including re ip • 
trican democracy there exists the hope that its ues 
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ing which would indicate that it is not entirely distinct from cr*^ 
cal thinking. First, we suspect that creativity as exemplified by 
Pascal, a Leonardo, a Marx, or a Picasso, is probably of such a higt 
and rare order as to be of little practical concern to general edu- 
cation teachers. Certainly, it would be unrealistic and uneconomi- 
cal to build a program for persons who occur so infrequently. The 
creative thinking which general education teachers must seriousl, 
consider might be roughly equated with originality in thinkii' 
and might well be developed on the part of all students comi ' 
under the influence of general education courses. Viewed as 
originality of thinking, creative thought becomes nothing more 
than the achieving of more spontaneous and less stereotyped solu- 
tions to a variety of problems. Whether or not other individuals 
at other times have solved the same problems in the same way is 
relatively unimportant. What is important is that students be given 
every opportunity to exercise and develop some individuality 
and originality in their thinking. 

In courses in which the instructor organizes the material and 
presents it to a largely passive audience of students, individuals 
other than the teacher have virtually no opportunity to demon- 
strate any originality or creativity in thinking. Problems are pre- 
sented to the student in the form of periodic quizzes which re- 
quire as solutions repetition of organization presented by teachers 
in the context of their lectures. The only creativity demanded is 
lat of creating a smaller facsimile of a larger body of content 
presented by the professor’s lecture. On the other hand, in a 
t ought-oriented course demanding student practice and demon- 
stration of a variety of the higher mental processes, students might 
eve op into highly original thinkers. In place of the professor 
e ming an area of subject matter and presenting his exegetical re- 
i! \ ^ ^c^cher would present problems frequently for which 
e\cn c '*^clf had no ready solution. Students would be required 
to so ler orally or in writing these problems, with the pre- 
mium eing p aced on accurate intellectual processes and spon- 
taneous, imaginative solutions resting firmly on a rational set of 
premises. 

An approach to teaching used by a noted botanist illustrates 
what we mean by creativity. He presented his students each with 
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The Status of Knowledge about Critical Thinking 
We have hypothesized that critical thinking involves mental proc- 
esses and a point of view which are much the same from one field 
to another. The extent to which this is so and even whether it is so 
needs to be explored by research. Further, the kind of research 
which is necessary before critical thinking can be definitely as- 
signed such a prominent place needs to be of a different order than 
that heretofore reported. Prior research has been carried on for 
the most part by psychologists, educators, and logicians operating 
quite independently. While the insights these individuals have 
gained are valuable and must provide the basis for future work, 
their various conceptions of critical thinking and the assumptions 
from which they operate have been more limited than is necessary 
if critical thinking is to become the central and integrating objec- 
tive of general education. 

Psychologists have attempted to infer mental process from the 
observed products of thought, and much of the experimental work 
has been done with respect to lower animals or quite young chi 
dren. The descriptions of trial-and-error behavior in animals, 
studies of insight in anthropoids, facing children with puzzle-type 
problems, and the examining of language of children in spon 
taneous talk, are illustrative of what has been done. Considerable 
work has also been carried on by using a variety of tests allowing 
the experimenter to consider not only the test results but the e- 
havior of subjects while taking the tests. All such approac es are 
subject to considerable error, for they all rest on the assumptio 
that the nature of a complex, internalized process can be inteireu 
from observation of relatively tew overt behaviors. Experimen e 
we always faced with the dilemma of acquiring more and mo 
information about their subjects to such an extent t at t n 
sultant resembles a psychoanalytic protocol for advancing i 
ences which are extremely questionable. nf 

Educators rely to a considerable extent on the metho ^ 
psychologists and also assume a correspondence 
products and processes. They generally assume tha 
Ewing the largest number of correct responses on an ex 
®re the individuals best endowed with desirable me 
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tions be subjected to constant scrutiny so that those who accept 
them do so reasonably and not by blind faith Knowledge acquired 


without reason becomes a mere jumble of facts and not the essen 
tial instruments of an educated man The techniques by which 
knowledge can be effectively utilized are those involved in critical 
thinking Individuals sympathetic to a student centered or to a 
needs point of view may decry major emphasis on critical thinking 
as a move which accepts a duahstic concept of the nature of man 
and places emphasis on intellectual outcomes This is not our posi 
tion Those who accept responsibility only for cultivation of the 


mind will view critical thinking as an intellectual process but 
others will see critical thinking as involving the total person A1 
though critical thinking may be interpreted somewhat differently 
according to the preconceptions of the individual teacher, it is 
these preconceptions that limit critical thinking rather than any 
limitation intrinsic to critical thinking itself Even so, adherents of 
these different vieivpomts should find a large area of common 
^und upon which they can work Critical thinking must be 
based on knowledge and the effectiveness of materials m provid 
mg expenence in critical thinking can actually become a criterion 
0 relevance in choosing materials from the various broad general 
c ucation areas Likewise it can become the integrating factor for 
eve oping comparisons and contrasts among tlie various areas 
e importance of critical thinking in citizenship and in adult 
1 e suggests that problems drarvn from life situations can be used 
extensive y Critical thinking cannot ignore values Rather, at 
r ^ cntical thinking should cause each student to examine 
an e me is own value position since it is irretrievably inter- 
wo\en into t e thinking he does and the solutions that he reaches 
ritica t iin mg can take place in the context of quite differing 
uc sptems ut thinking is not truly cntical unless it recognizes 
tlie existence of several value systems and the acceptance-too 
often unconscious-of one of them In fact, our proposal regarding 
CTitica! Uunking is not at all a revolutionary one. for the increased 
freedom of choice central to many discussions of the liberal arts 
imphcitlj requires ability to propose and analyze alternative ideas 
pnor to making a choice 
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ing is done unconsciously, a bias is introduced in the data so ac- 
cumulated which is almost impossible to ferret out. 

In addition to the weaknesses already mentioned, research to 
date in the area of critical thinking has been characterized by 
several other limitations. The fact that this research can be de- 
scribed under several disciplinary rubrics itself suggests that re- 
search has been too compartmentalized. The logicians have fol- 
lowed their bent, the psychologists have frequently been exclu- 
sively concerned with animal psychology, and the educators have 
moved in still another direction. It appears to us that the insights 
from all of these disciplines need to be brought together to supple- 
ment each other in providing an integrating base of research evi- 
dence from which other attacks on the problem may be launched. 

Much of the research accomplished to date has been divorced 
from teaching practice. In the case of retrospective inquiries, re- 
course has been made to the autobiographical writings of outstand- 
ing individuals. In the case of psychological investigation, much has 
been done with animal experiments and with highly artificial ex- 
periments far removed from conditions as they would be in a 
classroom. Much, for example, has been written about retention, 
but such experiments deal with the retention of nonsense material 
or discrete factual information, neither of which is ideally the 


major outcome of general education college classes. 

In this same connection there is a noticeable lack of suggestions 
and evidence as to how critical thinking can be taught or, for 
that matter, how any mental trait of adult humans can be de- 
''cloped through planned experiences. A survey of psyc o ogica 
Hterature for the past eight years reveals not more than a score 
nf articles which truly provide evidence on learning and the de- 
velopment of the higher mental characteristics. Almost any pnn e 
^■eview of psychological research suggests strongly that t * 
in which research should be undertaken, but to date 
noting has been done. , , . „:.hcr 

'Vhat research has been carried out and descn e 
Ps>chological, philosophical, or educational j of 

'I'^ntlybeen so couched as to repel rather than 
“ther kinds o£ courses. For example, some stud.es regarding lean. 
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IS obvious that such a conception is limited in usefulness, for until 
the educator can know and understand the relationship between 
student solutions to problems and the thought processes leading 
to those solutions, they are unable to determine effective from 


ineffective instructional procedures 
The logicians, with perhaps the longest history of disciplinary 
concern for thought processes, attempt to postulate thinking as a 
series of logical, rational steps The logicians have tended to talk 
in terms of ideal methodology by which the final goal or solution 
to a problem is reached with a minimum of effort by means of 
precise steps It is quite likely, however, that the majority of people 
do not solve problems in such a methodical way, and that most of 
the problems faced by individuals and by society cannot be 
solved in quite so straightforward a fashion While the logicians 
may provide for use of idealized schemes of thought and tools 
with which thinking processes may be taught, their insights may 
not be too relevant in investigating internalized thought processes 
Some insight into the thinking process has been achieved by 


various investigators through introspection or from narrative ac 
counts from individuals as to how they arrived at a particular con 
dusion after they had completed a process or experience Thus, 
Dewey reports on the ways in which he attacked certain problems 
and the conditions which led him to select one solution m place of 
anoi er. Einstein s reported thinking which led to his enunciation 
of the theory of relativity has also been described Certain auto 
biographical materials such as The Education of Henry Adams 
have been studied for ideas concerning thought processes This use 
of mtrospection and retrospection, however, in spite of being 
ighly interesung and provocative, has many major difficulties 
ersons find it hard, if not impossible, to report all of the major 
s eps m their thinking and to report them in a way in which they 
originally tKcumd The reporter frequently is inclined to edit 
and If he has been exposed to the Western tradition of logical 
h.nkmg, is mchned to portray h.s thinking as paralleling the 
logician s steps in thoughL Once a solution has been arrived at, the 
reporter frequently forget, the many dead ends he might have 
explored prior to making his decision Since so much of this edit- 
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of general education can meet. Before this meeting of minds can 
take place the validity of critical thinking for each position must 
be made explicit and be accepted by all teachers. 

Teachers ivho insist on a particular value orientation for gen- 
eral education might be sho^vn that teaching for critical thinking 
does not demand questioning basic religious assumptions. It im- 
plies only that these assumptions be recognized and that their role 
in the solution to life problems be made explicit. Those who hold 
that man’s intellect can be sharpened by close contact with the 
great minds from all ages, might be led to question to what extent 
this is true and whether or not some “great books might not be 
better than others as devices for this disciplinary function. Those 
who emphasize the primacy of knowledge may be re^sured that 
making critical thinking central need in no way minimize the 
importance of knowledge, and that one possible outcome of con 
centration on critical thinking might be acquisition of more 
knowledge in more meaningful form on the part of stu ents. 
Teachers who hold that general education should be 
toward more utilitarian ends should recognize that a pro em 
solving approach can be most effective in bringing education 


the level of student needs. , 

If a fairly large number of teachers became concerne enoug 
to give serious consideration to research in critical thin 
reconcile their differences, they are well on the way 
mg still a third need, which is to combine competence in 
fields of specialization with some awareness of and concern 
research findings and with the study and development o app 
^te instructional materials and methodology. cne- 

college teachers have been skilled in the field of t eir m 
cialization. They have assumed that ® been 

a good history teacher. Their consideration of resea 
for the most part a consideration of research in is o^- 

needed wL are willing to gtant that *0- knowkd^^e. 
O'vn subject does not automatically induce pro ci ^ needed 
'Og skills and abilities on the part of students, eac 
"|ho are sufficiently concerned about their ' -xperimenta- 
‘hey will gladly undertake any kind of jeid increased 

‘ion in connection with their oira classes which ^ y 
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mg have been carried out by educational psychologists in classes 
o£ educational psychology Teachers of other subjects are quite 
likely to regard such findings as being applicable, at most only to 
educational psychology Further, when teachers outside of the fields 
of psychology, logic and educational psychology refer to these re 
search findings, they find a technical jargon which defies interpreta 
tion by the uninitiated 

Still one further weakness m critical thinking research lies in the 
fact that the way it is presented too often suggests that emphasizing 
critical thinking can be a mechanical sort of operation tacked 
onto some particular college course Such an implication leaves 
teachers free either to reject attempts to teach for critical thinking 
as an improper and unbearable extra burden or to regard it as an 
extra frill requinng little time and no real modification of their 
heavy factual orientation 

Some Suggestions 

Having pointed out the lack of adequate research data to date 
ive are constrained to advance some ideas which we believe might be 
fruitful in the area of critical thinking Perhaps the first step in 
the development of major research m this area is for teachers to 
become concerned about the development of thinking on the part 
of their students We have repeatedly stressed m the preceding 
apters how cooperative consideration of evaluation problems in 
connection with the Cooperative Study made numbers of teachers 
suddenly aware that their classroom practices bore little relation 
s ip to t eir stated objectives So long as teachers are unaware 
or unconcerned about the apparent fact that imparting informa 
tion by means of class lectures or reading textbooks is relatively 
ineffectue m the development of the higher mental processes the 
importance of research in connection with critical thinking is 
hkely to be overlooked Means then must be discovered by which 
the need for research can be brought before teachers m all fields 
of general education 

Merely becoming aware of a lack m research about teaching 
techniques, however, docs not bridge important philosophic differ 
cnc« separating teachers Wc have suggested that critical thinking 
might serve as tlie common ground on which various viewpoints 
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the development o£ educational experiences selected in relation to 
stated objectives. 

There is evidence in this Study that teachers do vary greatly in 
their success as determined by the gains made by students on tests 
of critical thinking, but this project was not designed to isolate 
the differences in the teachers, in the classroom experience 
afforded by them, or in the means of motivation which resulted 
in these differential improvements. Only careful observation of 
classes over a period of time could yield reliable evidence on these 
matters. 

Coincidentally, it was found that in the case of teachers trying to 
modify their techniques for the avowed purpose of achieving better 
results in critical thinking, the differentials betsveen gains ma e 
by the experimental groups and those taught by the customa^ 
approach were commonly small. Moreover, these expenraenta y 
obtained individual gains svere often smaller than those ma e y 
students of other teachers in classes where no avowed special ettor 
was made with respect to the objectives. The implication is a 
minor changes in technique imposed upon a course w erein 
major emphasis remains on coverage of content are ina equa ® 
solution to the problem. The alternative conclusion that small 
gains are characteristic of such objectives and c 

instruction are of little importance seems less tenable beca 
the fact that students of some teachers apparently o ac 
sizable gains. Whether the differences in student achieve 
a result of what teachers do or what they mouvate q 
students to do is in itself an issue for research. aicriission 

The implication, to us, at least, of all the prece ' ^ 

m this chapter is that the identification of one major ] 
as critical thinking, the review of all researc o 
mental process, the observation and development of 

practice most conducive to effective learning, . j die ob- 

°ne or more integrated instructional ?' , major re- 

jective, and the evaluation of the results, cons teachers, 

search task. The attack should It should 

psychologists, evaluators, administrators, an s teacha- 

Provide information on the nature of the object 
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insight into teaching This need, incidentally, is not restricted to 
general education courses but characterizes all education as well 
It IS however, particularly appropriate to speak of it in connection 
with general education courses, for one can at least hope that 
teachers working with such course offerings for a period of years 
will come to view knowledge from a somewhat broader base than 
that provided by their own specialty In addition we can at least 
hope that general education will gradually attract persons trained 
along broader lines than customary graduate programs in some one 
or other of the college curricula 

These three needs seem to converge to underscore the major 
need of research in connection with critical thinking It is essen 
tial for such research to be basically oriented toward and integrally 
related to classroom practice By this we do not mean to suggest 
that certain kinds of philosophic and psychological research should 
not be encouraged It should be, but it should be undertaken to 
the end goal of providing insights for teachers It should be re 
garded as applied research having very definite practical goals in 
view Research findings when discovered need to be written in such 
a way that their implications for classroom teaching stand clear 
of aU fields^° described m terminology equally clear to teachers 


It must be apparent that the improvement of teaching and the 
evaluating of teaching effectiveness which are subjects of extensive 
study and discussion are enmeshed with the problem of teaching 
CTi tea t m mg Despite the extensive research on evaluation of 
eac mg ew conclusions of any significance have resulted, al 
^ ^ T variable is probably the most important one 
nrihJrni'^^F'k”^ Process Without minimizing the contributions 
nhlp ^ ^ school and community. It IS reason 

be foiinrl tn critenon of his teaching effectiveness is to 

be found m the extent to which his students achieve the objectives 
he leaches Unless one believes either that 
et^ are ho good teaching are synonymous or that good teach 
ers are bom and not made ’ there is some basis for believing that 
more adequate instruction with regard to specified objectives can 
result from more precise undentandmg of objectives and from 
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LIST AND AVAILABILITY OF MATERIALS 
DEVELOPED BY THE COOPERATIVE STUDY 
OF EVALUATION IN GENERAL EDUCATION 


I. Tests and Inventories 

A stock o£ the following tests and inventories has been deposited with 
the Educational Testing Service, 20 Nassau Street, i 

Jersey, and copies of them may be purchased from the erv 
as the supply lasts; 

Inventory of Beliefs, Form I, and Instructor s Manud ^ 
Problems in Human Relations, Form I, and Instruc o 
A Test of Critical Thinking, Form G, and Instructors 
A Test of Critical Thinking in Social Science, an 

A Vombilary Test for Use with A Test of Critical Thinking in 

Ten“of Sdence Reasoning and Understanding, Natural Sciences, 
Forms C and D, and Instructor’s Manual sciences, 

Test of Science Reasoning and Understanding, i S 

Te!t™ Science Reasoning and Understanding, Physical Sciences, 

Xerti. no instructor’s manual fo-; 

Physical Sciences tests, but the manual for Forms 

appropriate explanatory background.) . . instructor’s 

Test of Critical Analysis in Reading and wriii g 

A GuTdelo Critical Analysis and Judgment in the Humanities, and 
Instructor’s Manual. Four separate forms lor. 

Music 

Art 

Literature 

Philosophy . Humanities (for use 

A List of Abilities in Critical Judgment 
in scoring the student responses to the u / 

. 287 
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bility, on the qualities of good instruction and the extent to which 
teachers can acquire them, and on the learning processes of stu 
dents and the characteristics of individuals which facilitate or 
impede the development of critical thinking Comprehensive and 
complicated as such a research project would be, it is realistic in 
the same sense that the objectives instruction, materials, and 
techniques are all facets of the learning experience which must 
finally be evaluated by its effect on students 
This, we believe, is the task that general education must under 
take if It IS to bring classroom practice and student achievement 
into a reasonable correspondence with its objectives 



APPENDIX 


OBJECTIVE TESTS IN THE 
AREA OF COMMUNICATIONS 


The Drake University Test of Reading and Writing 
This test attempts to evoke from the student as many as 
the mental processes he should go through in close rea mg 
writing and to evaluate tlieir results The questions on ^ 

writing are interspersed to utilize a single passage to nv 
types of activity in the order that they occur in toa i g 
assumption behind the test is that an author’s use of sy 
arrangement is the means whereby he communicates ‘ 
intention, and that the perception of this ftsumotion is 

best index to that meaning and intention de 

that a high correlation exists between the ability o y 
saibe fully a piece of writing and the ability to 
employed as passages tor analysis are intended o ^ , ■plie test 
of the student's daily experience in written com “ , „ore 

IS untimed because it is believed that tests tor comprehen 
valid if the tune factor is omitted ^ t eight 

The test Items fall roughly into six large °tego^ importance of the 
to twenty tour items in each, depending "P™ test^more than one 
respective mental processes Some items probably tes 
category The categories are , ,, 

I The ability to organize material and to recognue relauon 

terns of development, and structures words and sentences 

2- The ability to recognize the semantic fo"®” ''^or the language 

v^hether the reference is to objective facts, or j 

« figurative, etc. „„ ,n sentences, nuances of 

5 The interpretation of words in context, meaning 


feeling, and shades of thought 

The recognition of soundness of argument including punctua 

The recognition of grammatical form and f 

^‘on . .fructure and 

The esthetic judgment of vord choice, rhythm, metap 
die whole 


passage 
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Humanities Parucipation Inventory, and Instructor s Manual 
Humanities Vocabulary Test (no manual available because of lack 
of data) 

II Rating Scales 

These scales may be reproduced and utilized by interested teachers 

Scale for Evaluating Speaking and Writing (see Figure 2, facing 
page 79) o % o 

Scale for Evaluating Speaking (see Fig 3, pages SO-Sl) 

III iNsniucnoNAL Handbooks 

The fdlowing three items are published by the Wm C Brown Com 
pany, Dubuque, Iowa 

Social Saence A Handbook for Evaluation and 
A Handbook for College 

Handbook for Theme Analysts 


IV Other Materials 

vo'lumTbJrnit 

the authon ^ Preceding lists should be directed to 
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OBJECTIVE TESTS 

Vocabulary meaning in context 
Comprehension of details, vocabulary relationship 
Vocabulary and allusions 
Association and inference 


Stephens College Test of Writing Ability, Form A1 
The Stephens College Test of Writing Ability is an untimed test of 
the student’s ability to recognize and evaluate important elements in 
writing and its revision. This test is based on the assumptions that the 
use of textual material which is interesting to the student creates a 
functional testing situation; that extensive passages of student and pro- 
fessional writing analyzed for improvement of communication or 
recognition of skill in communication will measure a student s unc 
tional knowledge of important elements in writing; and nna Y ^ ^ 
such a test will correlate more favorably with grades indepen en y 
assigned on actual writing than will grades on an English usage 
The sections of the test include items which measure such com 
elements of writing as structure, organization, level of usage, releya . 
reference, repetition, emphasis, style, punctuation, point o vi > 
velopment, transition, and word choice. 


The Brown-Carlsen Listening Comprehension Test 
The Brown-Carlsen Listening Comprehension 
through World Book Company) is a test of a student s abi y 
prehend material presented orally. It is machine or 

responses being recorded on IBM-type answer shee s, 
hand scoring. Administration time is under 50 minutes. 

The following aspects of listening are explored. ^ 

Test Items 

^ 20 

Test 1. Immediate Recall 

Test 2. Following Directions 2^ 

Test 3. Word Clues 

Part A: Recognizing Transitions 
Part B; Recognizing Word Meanings 
Test 4. Lecture Comprel.ension (10-minute lecture) ... 

Part A: Getting the Details 
Part B; Getting the Central Ideas 
Part C; Depth of Understanding 

In general, the 90 items making up the squiring re/7«- 

quinng an accurate reception of details, a , . q, an ccalu- 
“n on the details— a synthesizing, an inte . ‘ measure 

^“ng. Tests 1, 2, and Part A of Test 4 contain items 
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Michigan State College Reading Test 


The Michigan State College Reading Test is composed of 150 multi 
pie choice test items Sixty five of these are vocabulary and the re 
mainder are of a largely self contained type based upon reading pas 
sages, that is, no information outside the paragraph is needed to answer 
them The administration time is two hours, and it is anticipated that 
nearly all students will be able to finish within the allotted time In 
other w ords, it is not a time test 

The vocabulary items deal with words selected from the outlines 
of study of the basic courses at Michigan State College These items are 
of tuo types (I) those in which the words are given in the stems and 
the student is asked to select from the responses the word or phrase 
which IS most appropriate, and (2) those in which definitions are 
given in the stems and the student is asked to select the appropriate 
word from the responses Examples are 


Provinctal 

1 significant 

2 prejudiced 

3 narrow 

4 insignificant 


Anything that on organism 

1 attitude 

2 behavior 

3 motive 

4 personality 


speaking, the items in the reading portion seek to deter 
1 “'"‘‘“5’ ">d'n''e meaning from passages selected 

IZde A In'' 1 'Denve meanmg" would 

ences translating fi ^ synthesizing, drawing conclusions, making infer* 
about three Iiimflr etc A typical paragraph contains 

eight to ten items '°'*°wed by approximately 


STTPiicNS CocucE Text of Reading Compeehens.on 
lest of the 

meanings This test is basMo!i'’ir^‘* comprehension of precise 
for comprehension are Ce "'"'‘l."® 

textual material comnosed oF ^ ^ ““““ed. that 

a more realistic and'TunliLal sml"? 

posed of a series of oaramnt, ' *an textual material com 

a close analysis of *"Sa£ ,'57"''''“'"’ 

and relationships adcauatelv i ' reeognition of precise meanings 

T.. 

Recognition of them and purpose 

Author , baciground experience, point o[ view 

Comprehension Of mam ideas ^ 

^mprehension ot detail, the dranmg oI interences 
Tone and attitude of author “ «ences 
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Abilities See Critical abilities. Critical 
thinking— Abilities 
Activities inventory, 225 
Adequacy o! data, recognition, 39 
Adult life, preparation for, 271 
American Counal on Education Psycho- 
logical Tests, 181 
Angell, George, 10, 11 
Antioch College, 9 
Appropnateness, doctrine of, 84 
Art 

exhibits, attendance by students, 168 
See cho Combined arts, Fine arts. 
Painting, Plastic arts 
Assumptions 
criteria formulated by 
Cnucal Thinking Committee, 178, 
180 

Social Science Committee, 38 
definition, 180 
illustrative item, 188 
Attitudes, 208 41 

w outcomes of general education, 212 
assessing, technique for. 147 
changes m, evaluation, 148 
changing, techniques for, 241 
Compared with opinions, 146-47 
concepuon of Attitudes Committee, 
211-12 

cotena formulated by Science Com- 
mittee, 109 

definitions, 146, 210 11 
c5ect of teachers upon. 240-41 
” dignity of others, 229 50, 

high scorers and low scorers compared, 
225 

measurement of all dimensions, 212 
concern to general education, 215- 

*‘Smficance of projects, 239-41 


Attitudes Committee 

assumptions of working principles, 212 
membership, 208 
obicctne, 12 

relationship to other intercollege com- 
mittees, 238 

Attitude, Inventory S« 

Audio viiuiil techniquB, 70, 
Authoritarian atmosphere on coUeg 

aropuses, 241 

Authonly. evaluation of, 38 30 
Autocrat,, types of, 230 31 

Bailer, Warren R, The Cose o/AfieSef 

5s£SMr7i‘’7V73fror,-f7?2oo, 

situations 264 65 sneakine 

enective and tneSerUve, tn spealung 

qnesUom’foraSa'ed, 2125 

'rn^"ei'hehefs,22e,220 

conclusions tegajding, 2 

correlations with other te,«, 2 

development, ^ 

disagree and a^ j^dg 

four element key 

ments, 218 2 ^ 241 

gams of students. 226 28. 2.^, 

Kstmetor’s Manual, 222,28/ 

rehabilit), 222, ^9 

-r;r-S'ex“o%ersonah.y.truc 


293 



292 


GENERAL EDUCATION 


accuracy of reception Test 3 and Parts B and C of Test 4 contain 
Items which measure reflective aspects of the listening process 

Stephens College Test of Listening Comprehension 
The Stephens College Test of Listening Comprehension is an un 
timed test of the student’s ability to listen for comprehension of a 
speakers meanings in expository oral communication This test is 
based on the assumptions that textual materials composed of extensive 
excerpts from talks present a realistic functional situation common in 
the students daily experience, that two passages, one primarily ex 
p anatory, the other primarily argumentative, represent two very 
common listening situations, that testing a student’s recognition of 
mam ideas and details, logical inferences, and relationships of ideas 
wi pve a valid measure of a student’s general listening ability, and, 
na y, at pre testing and post testing of such general ability, with 
in en^mng instruction in listening, will yield a measure of growth in 
In processes occur more or less simultaneously 

out the*Lst'"^ situation, items testing them are interspersed through 
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Cntical Analysis in Reading and Writing 
Test— Confmucd 

gams of students, 96, 97 98, 99 100 
Instructors Manual, 287 
matenal in second test, 98 
reliability, 96 

research data from, 96 101 
student reactions, 97 
validity, 96 

See also Reading comprehension. 
Writing 

Criucal incident study, 70 85 
conclusion of Communications Com 
mittee, 85 

Critical thinking, 174 207 
abilities 

brief list. 179 81 
correlations, 191 92 
See also Critical abilities 
as principle of integration, 272 78 
citizenship 35 36 
common aspects, 275 
concept 36 
definition, 176 

disagreements as to nature and at 
tainability, 174 76 

involvement of the total person, 278 
knowledge about. 279 82 
meaning, 273 74 
objective 176 77, 267 
philosophic differences separating 
teachers, 282 83 

relation of teaching effectiveness to, 
284 

relation to intelligence, 56 
relation to other integrating concepts, 
277 78 

research, 279 86 
teacher awareness, 282 84 
suggestions of directors 282 86 
Tentauve List of the Problem Solving 
Aspects of Cntical Thinking, 177 
79 

S'e oijo Cnucal Thinking Test 
Critical Thinking Committee 
Membership, 174 
ohjectives 12 

Critical thinking m social science 
course patterns, effect of, 62 6S 
^rtltcal Thinking m Social Science 
d Handbook for Evaluation and 
Teaching, 65 66. 288 

of socioeconomic background, 
58 59 

®bjeciive of Social Science Committee, 
85 37 


Cntical thinking in social saence-Con 
tinued 

student growth and development, 

55 65 

teaching, collegiate, 64 65 
teaching techniques, effect of, 63 64 
Critical Thinking in Social Saence Test 
availability, 287 
omstruction, grid form for, 48 
correlations with other measures, 57 
58 

correlations with other tests, 56 57, 201 
gains of students, 59 65 
Instructor's Manual, 55, 287 
preparation, 41 49 
reliability, 55 

research data, summary of. 55 65 

student achtevement compared with 

that on vocabulary test, 56 57 
typical test items, 50 54 
validity, 55 

Cntical Thinking Test 

availability, 287 , * - «f. 

recording 

196 

Form B, 245 

earns of students 202 6 

®^;x,„du.iont 205, 206 

Instructor’s Manual. 28/ 

Item analysis 194 
reliability. 190 K 

specifications fcr a 

Ihinting. 183 8h recording 

student judgment, form tor reco 

student teacuoiis, 198. 207 
summary, 207 

valtduy. 192 98 


r/arfcipi-n rn communi.y 

I'ing. 252 

iSul’T«^62 
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Beliefs, Inventory ot—ConCinuid Conclusions 

shorter lonn of test. 222 criteria formulated liy 

students from Nonhern and Southern Critical Thinking Committee, 17 


univenilies compared, 226 
vaUdiiy. 222 24. 226 
working time to administer test, 221 
Bias, recognition of, 38 
Biological sacnces See Sacnce Reasoning 
and Understanding Test 
Brown Catlsen lutening comprehension 
test, 291 


Campus organization local, 15 
Carnegie Corporation of New York, vn 
Case method, 261 

Catholic colleges, integration, 268 69 
Central issues identincation, 38 
Chicago, Unnersity of, College 9 
Chubb Crater, 115, 117 
Citizenship 

classroom expenenees, 30 
cnlial thinking in. 35 36 
good 

as anobjectiie, 33,271 
deDnitions. 35 
itudies, 35n 

aichii recognition of, 38 
Colleges 

A, Z, and C compared 230 52 
partiapaung v. IS. 16 
effect of Study On, 233 55 
Combined arts 
media included, 166 
research results IG5 66 
Commmee on Measurement and Evalua 
lion. ii. xil, 11 

Committees, meeitngs, U 15 
“Common knowledge,* 27o 
Communications. C8 101 
criteria of success, 81 , 

objectiie tests, 91 90, 289 92 
research pcniibilities. 70, 83 
skills, integrating factor, 93 91, lOl 
summary and Imphcauons, 101 
See alio Reading comprehcruion. 
Speaking; \\ thing 
Communicatioiu Committee 
membership 65 
objeetises, 12, 69 

Community government, student par- 
ticipation in. 232 
Conant, James 29 
Concerts, attendance by ttudrou, 163 


180 81 

SciMice Committee, 108 9 
Social Saence Committee, 39-40 
illustrative item, 189 
Confidential use of data, 18 
Consistency, checking, 39 
Content oriented instruction, 4 5, 7, 8, 17, 
26 

Cooperative effort, need for, 262 
Cooperative Study of Evaluation in Gen 
eral Education 
contributions, 243 44 
effect on individuals and colleges, 253 
55 

finanang, vii 

list and availability of materials de 
velopcd, 287 88 
operating plan, 14 17 
opinions of diiecton, 243 
problem of instruction, 242 
procedure, 11 17 
project approach, 14 
title, significance of, 18 19 
Correlations 

between pre test scores and gams, 248 
colleges A, £, and C, 250 52 
hypothesu loiestigated, 249 
influence of school s program, 255 
See obo subheading Cotre\ahon(s) 
under mam subject headings 
Counseling, 275 
Courses 

adaptation to students, 19. 20 
required, student reaction, 258. 259 
CieaUie thinking 

Soaal Science Committee conclusions, 
37 

viewed as originality of thinking, 
275 77 

Creative work, 141 
Creativity, testing for, 44n 
Critical abilities 
lut, 38 40 
validity, 40-41 

see oiro Critical thinking— Ablbtles 
Critical Analysis and Judgment Test 
See Humanities— Guide to Critical 
Analysis and Judgment in the 
Humanities 

Critical Analysis in Reading and Writ- 
ing Test, 91 90, 287 
correlation with other tests, lOO-l 
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Handbooks— Confmutftl 
Science Heosoning and t/ndersfandmg 
A Handbook for College Teachers, 
110, 120, 122, 288 
See also Humanities 

Handicraft, engaging in, by students, 168 
Hell, Louis, 108 
Hook, Sidney, 3 

Human Relations Test See Problems in 
Human Relations Test 
Humanities, 139 73 
Attitudes Inventory, 148, 160 
attitudes toward, 145 49 
courses 

history, 139 40 
objectives, 142 43 

objectives and classroom practices 
compared, 170 72 
rethinking of role, 169 
types, 140 42 
creative work, 141 

cnucal analysis and judgment, 151 59 
cntical thinking as objective, 274 
divergent theories regarding nature of 
studies, 151 52 

Guide to Critical Analysis and Jodg* 
ment in the Humanities, 156 58, 
287 

acceptance of rationale, 170 
appraisal, 172 73 
Instructor 8 Manual 287 
teacher reactions, 170, 172, 259 
validity, 160 63 
handbook outlined 243 
List of Abilities in Critical Judgment 
in the Humanities, 153 56, 287 
use of, 158, 159 

research design, original, 169 70 
research results, 159 73 
unique quality of studies, 152 
Humanities Committee 
membership, 139 
objectives, 12, 142 

umanities Partiapation Inventory, 143 
45.160,288 

^rrelations with other tests, 163 64 
pms of students 166 69, 170 

instructor s Manual, 288 

udent experience at time of entering 
college. 164 66 

'^°“^«^2ry Test. 149 51. 
“rrelations with Humanities ParUa 


Hypotheses 

critcna formulated by 
Cntical Thinking Committee, 178 
180 

Science Committee, 108 
illustrative item, 189 


Individuals partiapatmg effect of 
Study on, 253 55 
Inferences 
definition, 39 
sample statement, 147 
validity, 180 

illustrative item, 189 90 
Information 

criteria formulated by 

Critical Thinking Committee. 17/ 
78 

Science Committee. 108 
illustrative item, 188 
Instruction See Teaching 
Integration 251 

Catholic colleges 268 69 
conclusions of directors, 266, 
correlation of Inventory of Beliefs 
With other tests 252 
courses, interrelation of, 252 
critical thinking 

Xr?^tS/»n«pts 277 78 
o'fC^Teoncepts 270 
72 

InteuSuaUy^onented instruction. 5 6 

*”“S5^^^^sona^A,^u“.Sen“' 
See Attitudes “"""““'communica 
Intercollege Co™™"" cknnmunica 
tions Objectives See tai 

uons Committee Critical 

Intercollege ''°“'““r,,ncal Thinking 
Thinking See Critical 

Committee Humanities 

intercollege Commit Com 

Objectives i>ee nu 
mittee Science Ob 

Intcrcollege “ committee 

jecuves see science 

Intercollege 5 [ socnce Com 

Objectives See socui 

See Correlations 

Intercorrelations 
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INDEX 


Diedench, Paul 94 

Difference in individual resulting from 
education, 28 
Discussion 
method, use of, 261 
proficiency in handling, 30 
Drake University Test of reading and 
Vfriting, 289 


Eckert. Ruth, 25, 28 
Education 

for the whole individual, 208 10 
Higher Education for American De 
mocracy, 1, 102n, 209n 
Educational experience, variations in, 
47 

Educational Testing Service, 10 
cooperation in Study, xii 
Elective system 268 
Emotional facton, recognition 38 
Essay materials 

approach to validauon, 123 24, 125 26 
interests of Science Committee, 120 
use by science teachers. 121 23 
See also Tests— Essay 
Evaluation 

characteristics of process 21 24 
concern about, 9 1 1 
Cooperative Study in General Edu 
ation, 10 

Evaluation in General Education 9 
Evaluation Staff of the Eight Year 
Study of the Progressive Educa 
lion Association, 10 
issues outlined by Ruth Eckert 25 
need for, 7 9 
purposes 19 21 

questions to be faced in entering into. 

reasons for Study, 1 17 
significance of, 29 
See also Teachers-Evaluauon 
Evidence, evaluation, 38 39 
Experimentation, need for, 7 9 260 61 
Experts, resort to, 231 32 


F scale, 220 

Facts, essenual. determination of, 39 
Faculty See Teachers * 

Fine arts, course objective, 142 

Flanagan, John C , 71, 84 

Forgetung of knowledge, when it begins. 


Gams of students 
caution in interpreting, 245 
characteristics of institutions with 
large and small gams compared, 
246 

data, reasons incomplete, 245 
differences in achievement, 285 
hypotheses for further research, 247 
49 

ranks of four colleges, 245 47 
summary of findings, 243 52, 266 
See also Beliefs, Inventory of. Critical 
Analysis in Reading and Writing 
Test Critical Thinking in Social 
Science Test Critical Thinking 
Test, Humanities Participation 
Inventory, Problems in Human 
Relations Test, Science Reason 
mg and Understanding Test— 
(Biological Sciences), (Natural 
Saences), (Physical Saences) 
General Aptitude Test Battery, 182 
General education 
defmiiions, 1, 34 
objectives, 2*4 
at practiang level, 13 
clarification, 19 

conclusions of directors. 265-67 
President’s Commission on Higher 
Education, 209 

reexamination of role, 263 66 
writing for catalogues, 263 
type of person as outcome. 216 
General Educauonal Development Tests 
College Level 181 
High School Level, 57 
Cenerahiation, determining whether 
facts support, 39 
Gestalt school of psychology, 208 
‘ Great books,” use in teaching, 140, 283 
Group dynamicists, research between 
communication workers and, 70 
Guide for Construction of Evaluation 
Items See Science 

Guide to Critical Analysis and Judg 
xnent in the Humanities See Hu 
mamues 


Handbooks 

CrtUeal Thinking m Social Science 
A Handbook for Evaluation and 
Teaching, 65 66, 288 
Handbook for Theme Analysis, 90 91, 
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Painting— Continued 
research results 162 63, 164 
Partiapation See Humanities Participa 
non Inventory 

Pcnnsyhama Oallege for Women, 193, 


200 


Perlman James S , 136 
Personal adjustment, 210 
Penonahty 
aids in defining 215 
The Authoritarian Personality, 216 
lour levels of personal involvements, 
21618 

types 250 32 
Philosophy 

discussion of topics by students, 168 
Guide to Cntical Analysis and Judg 
ment in the Humanities, 156 58, 
287 

reading of essays by students, 168 
research results 164, 165 66 
yrical sciences See Saence Reasoning 
and Understanding Test 
Pittsburgh meeting, 11 
Pjaceoent data, 225 

arts, research results, 165 66 


tendance at, by students, 168 
students, 168 
dents Commission on Higher 
'ation In 302 

Pr£^” ® general education, 20 < 
‘^wjlcra materials 
Jj«rative itexiv JS7 S8 
*eiencc field, los 

“«=na for sclec 

’“'“'I'onioUing 

If education, 21 

1T9 !1 ‘•“'*‘"8 =>>>'1 

f'WtKei ISO 

275 

15 '^“ &‘“cnl ThmUng, 

■\ si" Relation, Test 

“‘Rer tests 235 

i-feiRi,', '”">'.287 
252 33 

^■>2 I * 


Progressive Education Association, 10, 
182 

Psychologists, methodology, 279 

Questions to be faced in entering into 
an evaluation of general education, 
21 23 

Radio, listening by students, 168 
Rating scales 

for evaluating speaking, faang 79, 80 
8f, 8S, 288 

for evaluating writing, facing 79, 83, 
288 

Rationale of Study, 18 32 
Rattigan, B T , 268 69 
Reading comprehension 

critical thinking as objective, 274 
Drake University test, 289 
material m second test, 98 
Michigan State College test, 290 
Stephens College test, 290 91 
testing, 69, 94, 99 

See also Critical Analysis in Reading 
and Writing Test, Science ma 
tenals reading comprehension 
Reading student papers m search for 
previously determined responses, 158 
Regional cooperative projects, 262 
Relevance, principle of, 261 
Relevancy, determination of, 39 
Religious affiliation, 225 

sssujJipiJSMJS, 2BS 

Research 

between communications workers and 
group dynamicists, 70 
hypotheses for, 247 53 
implications for classroom teaching, 
284 

involving human beings, 24 
See also subheading Research under 
mam subject headings 
Rexroad, Carl, 194 

The Rubdiydt, objective questions on, 
94 96 

Sanford, R Nevitt, 220 
Saence, 102 38 
counes, content, 105 7 
Guide for Construction of Evaluation 
Items, 108 10. 120 
gullibility of students, 105 
objective pertinent m general educa 
tjon, 102 3 
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Interpenonal situations, approach to 
solutions, 230 35 

Inventones 

list and availability, 287 88 
See also Activities inventory. Beliefs, 
Inventory of Humanities— Attitudes 
Inventory, Humanities Participation 
Inventory 

Iowa High School Content Examination, 
200 


Kansas State College, 197 
Kuder Preference Record, 225 
Kuder Preference Record— Vocational, 
163 64 

Kuder Richardson approach to test te 
habihty, 234 

Kuder Richardson Formula No 20 
124. 191 


Lag between research results and prac 
lice. 24 

Laissez faire personality, 231 
Leadership, 226 
Learning process 
nature of 175 
research on, 281-82 
Liaison ofRcers, v, 16 

List of Abilities m Critial Judgment 
In the Humanities See HumaniUes 
Lutemng comprehension 
Brown Catlsen test 291 
critical thinking as objective 274 
evaluaung 70 
Stephens College lest, 292 
Literature 

Guide to Critical Analyiu and Judg 
ment in the Humanities, 156 58, 287 
on general education, use of 255 
r“«rch results, 161. 162 63. ’l64. 165 

Logic 
coune m 

correlation with Cntical Thinkma 
Test, 201 * 

objective, 206 
transfer value, 132 
Logiaans, methodology, 280 


McConnell T R., 1,8.9 
McGrath, Earl J , 8, 9, 10 


Measurement 
entena, 84 
means, 19 20 
of attitudes, 212 
Medical education, 269 
Meetings of intercollege groups, 262 
Mental Abihties, Primary, Thurstonc, 
182 

Michigan State College, 9, 200 
Miller Analogies Test, 182 
Minnesota, University of. General Col 
lege, 9 

Morse, Horace, 254 
Motion pictures 

frequency of attendance by students, 

167 68 

study of, 141 
Music 

classical, listening by students, 168 
Guide to Cntical Analysis and Judg 
ment in the Humanities, 156 58, 
287 

playing of instruments by students 

168 

research results, 161, 162 65, 164, 165 
66 

wnung by students, 168 

Natural sciences See Saence Reasoning 
and Undentandmg Test 
Newman, John Henry, 101 
Novels, reading by students, 167 

ObjecUve type tests See Tests 
Objectives 

criterion of teaching effectiveness, 284 
five steps suggested by directors, 265 
66 

relation of Study committees to, 12 
relationship to classroom practice, 266 
See also subheading Objectives under 
mam subject headings 
Operas, attendance by students, 168 
Opinions 

as indicators of penonality, 215, 216 
compared with attitudes, 146-47 
Oral communication See Speaking 

Painting 

'"gaging in. by students 168 
Guide to Cntical Analysis and Judg 
ment in the Humanities 156 58, 
287 


quesuons, objecuve, 159 
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research results, 162 63, p-rticipa 

PaiUapation See Humanities 
lion Inventory 193, 

Pennsyh’ania College for 

200 

Perlman, James S , 136 
Personal adjustment, 210 
Personality 

aids in defining, 215 *.g 

The Authorttanan nts 

four levels ot personal mvolsc 
216 18 
types, 230 32 

oE »p.c by student, 

Guide to Cnncal Analyst, nnd Ju | 
ment m the Humanities, 156 5». 

reading of essays hy students, 168 
research results, 164, 165 66 
Physical saences See Saence 
and Understanding Test 
Pittsburgh meeting, U 
Placement data, 225 .-kaa 

Plastic arts, research results, 16 

attendance at, by students, 168 
paruapauon in. by her £du 

Presidents Commission on Hig 
cation. In 102 
goals for a general educatSo . 

Pioblem materials 
lUusuauve item, 187 88 

lacncc field, 108 selection, 

soaal saence, entena ' 

45*46 

Problem solving , 285 

approach to general abihues. 

bnci list o! cnucal thtnUng 
119 81 

processes, 156 

soaal saence teaching, Zv» ^y,ng 
Tentamelast t"' 177 
Aspects of Cnucal T 
79 . 'Tf^t 229 

Problems in Human Relations 

38. 287 ^ 255 37 

correlations with other 
gams of students. 25< * 

Instructor’s Manual. -8 
item iclcaion. 232 33 
reUabihiy, 233 35 
tconng Veys, 233 34 
validity, 235 
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